Jul 29 17:52:28.613251 ip-10-0-129-228 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 17:52:29.022502 ip-10-0-129-228 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:52:29.022502 ip-10-0-129-228 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 17:52:29.022502 ip-10-0-129-228 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:52:29.022502 ip-10-0-129-228 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 17:52:29.022502 ip-10-0-129-228 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 17:52:29.023219 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.022559    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 17:52:29.026900 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026883    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:29.026900 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026899    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026903    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026907    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026911    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026914    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026917    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026919    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026922    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026924    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026927    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026930    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026932    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026935    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026937    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026940    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026942    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026965    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026968    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026971    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026974    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:29.026981 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026976    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026979    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026981    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026984    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026988    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026993    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026996    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.026999    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027002    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027004    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027007    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027011    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027015    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027018    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027021    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027024    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027026    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027029    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:29.027456 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027031    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027034    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027036    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027039    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027041    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027044    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027047    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027049    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027052    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027054    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027058    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027061    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027063    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027066    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027069    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027071    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027075    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027077    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027080    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027082    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:29.027927 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027085    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027087    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027090    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027093    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027095    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027098    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027100    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027103    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027106    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027108    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027111    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027113    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027116    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027118    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027121    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027123    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027126    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027129    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027131    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027134    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:29.028454 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027137    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027140    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027143    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027145    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027148    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027150    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027155    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027158    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027162    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027165    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027168    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027171    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027173    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027176    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027179    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027181    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027184    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027187    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027190    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027192    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:29.028965 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027906    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027913    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027917    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027920    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027923    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027927    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027930    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027933    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027936    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027940    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027944    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027947    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027950    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027971    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027974    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027977    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027980    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027982    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027985    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027988    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:29.029452 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027991    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027994    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027996    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.027999    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028002    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028004    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028007    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028009    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028012    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028014    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028017    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028020    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028023    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028025    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028028    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028030    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028033    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028035    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028038    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028040    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:29.029969 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028043    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028045    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028048    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028050    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028053    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028055    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028059    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028061    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028064    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028066    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028068    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028071    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028075    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028078    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028081    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028083    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028086    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028089    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028091    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:29.030485 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028093    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028097    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028101    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028103    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028106    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028111    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028114    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028117    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028120    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028122    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028125    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028127    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028130    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028133    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028135    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028138    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028141    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028143    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028146    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:29.030966 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028148    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028151    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028154    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028156    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028159    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028162    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028167    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028170    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028172    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028175    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028177    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028180    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028182    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028186    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028189    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028192    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028195    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028197    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028200    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028202    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:29.031481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028205    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028571    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028576    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028579    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028582    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028584    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028587    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028590    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028592    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028597    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028600    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028603    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028607    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028612    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028615    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028618    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028620    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028623    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028627    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:29.031991 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028630    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028632    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028635    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028638    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028640    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028643    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028645    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028648    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028650    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028653    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028655    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028658    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028660    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028663    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028665    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028668    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028670    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028673    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028675    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:29.032481 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028678    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028680    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028683    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028688    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028691    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028693    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028695    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028698    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028701    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028703    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028706    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028708    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028712    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028714    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028717    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028719    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028721    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028724    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028726    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028729    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:29.032967 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028731    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028734    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028736    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028739    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028742    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028745    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028747    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028749    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028752    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028754    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028757    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028759    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028761    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028764    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028766    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028772    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028775    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028778    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:29.033545 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028781    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028783    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028786    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028789    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028793    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028796    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028800    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028803    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028807    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028809    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028812    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028814    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028817    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028820    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028822    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028825    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028827    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028830    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028832    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028835    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:29.034015 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028837    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028840    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028842    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.028844    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028917    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028924    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028930    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028934    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028938    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028943    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028947    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028964    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028968    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028971    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028974    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028978    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028982    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028985    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028989    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028992    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028995    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.028998    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029002    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029005    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 17:52:29.034484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029008    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029011    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029014    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029018    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029021    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029023    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029027    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029030    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029033    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029035    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029038    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029041    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029045    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029048    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029051    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029053    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029056    2580 flags.go:64] FLAG: --enable-server="true"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029060    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029065    2580 flags.go:64] FLAG: --event-burst="100"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029068    2580 flags.go:64] FLAG: --event-qps="50"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029071    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029074    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029077    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029080    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 17:52:29.035111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029083    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029087    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029091    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029094    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029097    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029099    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029102    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029105    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029108    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029111    2580 flags.go:64] FLAG: --feature-gates=""
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029114    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029117    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029120    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029123    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029126    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029128    2580 flags.go:64] FLAG: --help="false"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029131    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029134    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029137    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029140    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029143    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029147    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029150    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 17:52:29.035689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029152    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029155    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029158    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029161    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029164    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029167    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029170    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029172    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029175    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029178    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029181    2580 flags.go:64] FLAG: --lock-file=""
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029185    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029188    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029191    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029197    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029199    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029202    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029205    2580 flags.go:64] FLAG: --logging-format="text"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029207    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029211    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029214    2580 flags.go:64] FLAG: --manifest-url=""
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029217    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029221    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029224    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029228    2580 flags.go:64] FLAG: --max-pods="110"
Jul 29 17:52:29.036306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029231    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029234    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029237    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029240    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029243    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029245    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029248    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029255    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029259    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029261    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029264    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029267    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029273    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029276    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029279    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029282    2580 flags.go:64] FLAG: --port="10250"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029285    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029288    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0772eb0a6e581f376"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029291    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029295    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029298    2580 flags.go:64] FLAG: --register-node="true"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029301    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029305    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029308    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 29 17:52:29.036920 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029311    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029313    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029317    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029320    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029323    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029325    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029328    2580 flags.go:64] FLAG: --runonce="false"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029331    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029334    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029337    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029340    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029343    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029346    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029349    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029352    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029355    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029358    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029361    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029363    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029366    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029369    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029372    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029377    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029380    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029382    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 17:52:29.037533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029386    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029389    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029393    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029396    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029399    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029402    2580 flags.go:64] FLAG: --v="2"
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029406    2580 flags.go:64] FLAG: --version="false"
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029410    2580 flags.go:64] FLAG: --vmodule=""
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029413    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.029417    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029536    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029540    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029543    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029546    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029549    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029552    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029555    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029557    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029560    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029562    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029565    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029568    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:29.038167 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029571    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029574    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029578    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029581    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029584    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029586    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029589    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029593    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029596    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029599    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029602    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029605    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029608    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029611    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029614    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029617    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029619    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029622    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029624    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:29.038735 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029628    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029630    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029633    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029635    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029638    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029640    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029643    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029645    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029648    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029650    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029653    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029655    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029658    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029660    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029662    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029665    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029668    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029670    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029673    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:29.039268 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029676    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029679    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029682    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029685    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029688    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029690    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029693    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029696    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029699    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029701    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029704    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029707    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029709    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029713    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029716    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029718    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029720    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029723    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029725    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029728    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:29.039731 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029730    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029733    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029735    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029738    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029740    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029743    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029745    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029748    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029751    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029753    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029756    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029759    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029761    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029763    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029766    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029769    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029771    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029774    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029776    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029779    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:29.040239 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029781    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029784    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029787    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029789    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029792    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029796    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029798    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029801    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.029803    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.030824    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.037808    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.037826    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.037845    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.037854    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037922    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:29.040720 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037927    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037930    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037933    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037936    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037939    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037942    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037945    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037947    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037970    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037975    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037977    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037980    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037989    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037993    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037996    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.037999    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038002    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038004    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038007    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038009    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:29.041113 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038012    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038014    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038017    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038019    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038022    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038024    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038027    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038030    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038039    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038042    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038045    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038048    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038050    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038053    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038056    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038058    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038061    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038063    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038066    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038069    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:29.041584 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038071    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038074    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038076    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038079    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038081    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038084    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038086    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038089    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038092    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038094    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038097    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038099    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038102    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038104    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038107    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038109    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038111    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038114    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038118    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:29.042085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038122    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038126    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038134    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038137    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038140    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038143    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038145    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038148    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038151    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038153    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038156    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038158    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038161    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038164    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038166    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038169    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038172    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038174    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038177    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:29.042561 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038179    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038182    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038185    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038187    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038190    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038192    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038195    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038198    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038200    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038204    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038207    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038210    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038214    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038217    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038220    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038222    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038232    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038235    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038239    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:29.043137 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038243    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.038248    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038397    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038403    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038406    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038409    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038411    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038414    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038416    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038419    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038421    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038423    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038426    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038428    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038431    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 17:52:29.043618 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038434    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038436    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038438    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038441    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038444    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038447    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038449    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038452    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038454    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038458    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038462    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038465    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038468    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038471    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038474    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038478    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038481    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038484    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038487    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 17:52:29.044010 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038490    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038492    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038495    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038499    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038502    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038505    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038508    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038511    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038513    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038516    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038518    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038521    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038523    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038526    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038528    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038532    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038534    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038536    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038539    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 17:52:29.044472 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038542    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038544    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038546    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038549    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038552    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038556    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038559    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038561    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038564    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038567    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038569    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038572    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038575    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038577    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038580    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038582    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038585    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038587    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038589    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 17:52:29.044924 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038592    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038594    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038597    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038599    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038602    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038605    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038607    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038610    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038612    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038615    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038617    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038620    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038622    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038625    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038627    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038629    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038632    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038634    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038638    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038640    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 17:52:29.045403 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038643    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038645    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038648    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038650    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038653    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038655    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038658    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038660    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:29.038663    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.038667    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.038813    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.041026    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.043058    2580 server.go:1019] "Starting client certificate rotation"
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.043159    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 17:52:29.045896 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.044005    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 17:52:29.046346 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.046332    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 17:52:29.048196 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.048179    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 17:52:29.053461 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.053439    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 29 17:52:29.056170 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.056156    2580 log.go:25] "Validated CRI v1 image API"
Jul 29 17:52:29.056968 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.056938    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 17:52:29.059187 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.059167    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 92c1040b-0aac-4157-8be5-750c5b8c5b0e:/dev/nvme0n1p4 b2b164b0-752f-43e5-8a73-fdda832ef595:/dev/nvme0n1p3]
Jul 29 17:52:29.059262 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.059185    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 17:52:29.063435 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.063330    2580 manager.go:216] Machine: {Timestamp:2026-07-29 17:52:29.062798896 +0000 UTC m=+0.332297669 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099569 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21a84ec75118f8333085efb483e1fa SystemUUID:ec21a84e-c751-18f8-3330-85efb483e1fa BootID:7e498a98-bc51-4b8b-b89a-5d7ba0d10fae Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:64:74:c0:4c:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:64:74:c0:4c:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:d0:13:14:d0:f5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 17:52:29.063435 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.063433    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 17:52:29.063550 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.063506    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 17:52:29.066661 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.066631    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 17:52:29.066808 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.066664    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-228.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 17:52:29.067001 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.066984    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 17:52:29.067967 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.067942    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 17:52:29.068007 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.067970    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 17:52:29.068007 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.067978    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 17:52:29.068791 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.068781    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 17:52:29.069633 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.069623    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 17:52:29.071637 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.071626    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 17:52:29.071760 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.071751    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 17:52:29.071789 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.071783    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 17:52:29.071816 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.071798    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 17:52:29.071851 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.071822    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 17:52:29.071851 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.071836    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 29 17:52:29.071851 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.071847    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 17:52:29.072405 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.072390    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 17:52:29.072484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.072409    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 17:52:29.074922 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.074901    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 17:52:29.077023 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.077009    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 17:52:29.077080 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.077039    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 17:52:29.078883 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078871    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 17:52:29.078967 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078887    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 17:52:29.078967 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078893    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 17:52:29.078967 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078898    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 17:52:29.078967 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078907    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 17:52:29.078967 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078913    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 17:52:29.078967 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078923    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 17:52:29.078967 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078931    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 17:52:29.078967 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078939    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 17:52:29.078967 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078945    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 17:52:29.078967 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078971    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 17:52:29.079241 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.078977    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 17:52:29.080538 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.080527    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 17:52:29.080538 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.080538    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 17:52:29.081679 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.081658    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 17:52:29.081741 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.081699    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-228.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 17:52:29.083858 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.083845    2580 server.go:1268] "Started kubelet"
Jul 29 17:52:29.083948 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.083923    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 17:52:29.084410 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.084335    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 17:52:29.084410 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.084401    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 17:52:29.084948 ip-10-0-129-228 systemd[1]: Started Kubernetes Kubelet.
Jul 29 17:52:29.086986 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.086881    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 17:52:29.088213 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.088194    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 17:52:29.088582 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.088561    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 17:52:29.092564 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.092547    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-228.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 17:52:29.093559 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.092567    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-228.ec2.internal.18c6d422ffc1509d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-228.ec2.internal,UID:ip-10-0-129-228.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-228.ec2.internal,},FirstTimestamp:2026-07-29 17:52:29.083816093 +0000 UTC m=+0.353314866,LastTimestamp:2026-07-29 17:52:29.083816093 +0000 UTC m=+0.353314866,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-228.ec2.internal,}"
Jul 29 17:52:29.094175 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.094160    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 17:52:29.094590 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.094574    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 17:52:29.094680 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.094661    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:29.095208 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095194    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 17:52:29.095379 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095362    2580 factory.go:55] Registering systemd factory
Jul 29 17:52:29.095446 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095383    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 29 17:52:29.095446 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095401    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 17:52:29.095446 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095414    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 17:52:29.095558 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095478    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 17:52:29.095558 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095488    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 17:52:29.095619 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095594    2580 factory.go:153] Registering CRI-O factory
Jul 29 17:52:29.095619 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095605    2580 factory.go:223] Registration of the crio container factory successfully
Jul 29 17:52:29.095680 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095646    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 17:52:29.095680 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095669    2580 factory.go:103] Registering Raw factory
Jul 29 17:52:29.095740 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.095682    2580 manager.go:1196] Started watching for new ooms in manager
Jul 29 17:52:29.096176 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.096165    2580 manager.go:349] Starting recovery of all containers
Jul 29 17:52:29.097293 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.097264    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 17:52:29.097392 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.097369    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-228.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 17:52:29.105156 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.105040    2580 manager.go:354] Recovery completed
Jul 29 17:52:29.110108 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.110095    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:29.111521 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.111503    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zbmt2"
Jul 29 17:52:29.113216 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.113200    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:29.113277 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.113238    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:29.113277 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.113252    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:29.113808 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.113792    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 17:52:29.113808 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.113805    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 17:52:29.113932 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.113823    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 17:52:29.113932 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.113837    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:29.114920 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.114856    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-228.ec2.internal.18c6d4230181f481  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-228.ec2.internal,UID:ip-10-0-129-228.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-228.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-228.ec2.internal,},FirstTimestamp:2026-07-29 17:52:29.113218177 +0000 UTC m=+0.382716956,LastTimestamp:2026-07-29 17:52:29.113218177 +0000 UTC m=+0.382716956,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-228.ec2.internal,}"
Jul 29 17:52:29.115653 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.115639    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:29.115712 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.115664    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:29.115712 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.115674    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:29.115899 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.115885    2580 policy_none.go:49] "None policy: Start"
Jul 29 17:52:29.115935 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.115903    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 17:52:29.115935 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.115914    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 17:52:29.117168 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.117151    2580 policy_none.go:47] "Start"
Jul 29 17:52:29.117573 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.117555    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zbmt2"
Jul 29 17:52:29.145817 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.145800    2580 manager.go:337] "Starting Device Plugin manager"
Jul 29 17:52:29.145989 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.145862    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 17:52:29.145989 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.145876    2580 server.go:87] "Starting device plugin registration server"
Jul 29 17:52:29.146185 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.146171    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 17:52:29.146262 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.146188    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 17:52:29.146308 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.146285    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 17:52:29.146539 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.146369    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 17:52:29.146539 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.146382    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 17:52:29.147029 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.146997    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 17:52:29.147107 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.147053    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:29.162902 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.162884    2580 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2613 into the system container \"/system.slice\": <nil>"
Jul 29 17:52:29.246512 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.246484    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:29.247912 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.247458    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:29.247912 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.247489    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:29.247912 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.247499    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:29.247912 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.247525    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.255557 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.255540    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.255617 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.255564    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-228.ec2.internal\": node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:29.264409 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.264379    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 17:52:29.265786 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.265771    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 17:52:29.265851 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.265809    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 17:52:29.265851 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.265830    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 17:52:29.265935 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.265861    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 17:52:29.268021 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.268002    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:29.268750 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.268734    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:29.366196 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.366094    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-228.ec2.internal"]
Jul 29 17:52:29.366196 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.366191    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:29.368200 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.368183    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:29.368270 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.368214    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:29.368270 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.368225    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:29.369277 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.369261    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:29.369788 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.369776    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:29.369909 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.369897    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.369995 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.369928    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:29.370574 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.370552    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:29.370652 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.370580    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:29.370652 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.370589    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:29.370652 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.370552    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:29.370652 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.370647    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:29.370777 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.370657    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:29.371978 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.371946    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.372032 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.371993    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 17:52:29.372864 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.372849    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 17:52:29.372930 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.372892    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 17:52:29.372930 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.372903    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeHasSufficientPID"
Jul 29 17:52:29.397621 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.397588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d9f93c40a3798d4828fdeedd9a7ffca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal\" (UID: \"4d9f93c40a3798d4828fdeedd9a7ffca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.397621 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.397619    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d9f93c40a3798d4828fdeedd9a7ffca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal\" (UID: \"4d9f93c40a3798d4828fdeedd9a7ffca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.397804 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.397635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d14432099af8323069e39bbc1aa22244-config\") pod \"kube-apiserver-proxy-ip-10-0-129-228.ec2.internal\" (UID: \"d14432099af8323069e39bbc1aa22244\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.400727 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.400705    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-228.ec2.internal\" not found" node="ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.405115 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.405097    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-228.ec2.internal\" not found" node="ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.469874 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.469845    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:29.498335 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.498308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d9f93c40a3798d4828fdeedd9a7ffca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal\" (UID: \"4d9f93c40a3798d4828fdeedd9a7ffca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.498421 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.498330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d9f93c40a3798d4828fdeedd9a7ffca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal\" (UID: \"4d9f93c40a3798d4828fdeedd9a7ffca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.498421 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.498376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d9f93c40a3798d4828fdeedd9a7ffca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal\" (UID: \"4d9f93c40a3798d4828fdeedd9a7ffca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.498421 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.498394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d14432099af8323069e39bbc1aa22244-config\") pod \"kube-apiserver-proxy-ip-10-0-129-228.ec2.internal\" (UID: \"d14432099af8323069e39bbc1aa22244\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.498518 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.498446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d14432099af8323069e39bbc1aa22244-config\") pod \"kube-apiserver-proxy-ip-10-0-129-228.ec2.internal\" (UID: \"d14432099af8323069e39bbc1aa22244\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.498518 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.498462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d9f93c40a3798d4828fdeedd9a7ffca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal\" (UID: \"4d9f93c40a3798d4828fdeedd9a7ffca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.570540 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.570508    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:29.671203 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.671141    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:29.703682 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.703653    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.708729 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:29.708714    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:29.771749 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.771719    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:29.872282 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.872247    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:29.972806 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:29.972723    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:30.042312 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.042280    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 17:52:30.042915 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.042436    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 17:52:30.073644 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:30.073614    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:30.121807 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.121747    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 17:47:29 +0000 UTC" deadline="2028-02-23 01:07:45.378891085 +0000 UTC"
Jul 29 17:52:30.121807 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.121797    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13759h15m15.257098194s"
Jul 29 17:52:30.174131 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:30.174096    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:30.274196 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:30.274169    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:30.274196 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:30.274167    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14432099af8323069e39bbc1aa22244.slice/crio-fb5bf88312f299b4e5f13b540f8d92ca5ecdc38691a560b817c3e65e9abd0f7a WatchSource:0}: Error finding container fb5bf88312f299b4e5f13b540f8d92ca5ecdc38691a560b817c3e65e9abd0f7a: Status 404 returned error can't find the container with id fb5bf88312f299b4e5f13b540f8d92ca5ecdc38691a560b817c3e65e9abd0f7a
Jul 29 17:52:30.274610 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:30.274586    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d9f93c40a3798d4828fdeedd9a7ffca.slice/crio-7b45af92f944f03f80c352dc28ee110eb160b4e9200459fbdd8ce372d5889a44 WatchSource:0}: Error finding container 7b45af92f944f03f80c352dc28ee110eb160b4e9200459fbdd8ce372d5889a44: Status 404 returned error can't find the container with id 7b45af92f944f03f80c352dc28ee110eb160b4e9200459fbdd8ce372d5889a44
Jul 29 17:52:30.278415 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.278395    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 17:52:30.375132 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:30.375089    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:30.475690 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:30.475663    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-228.ec2.internal\" not found"
Jul 29 17:52:30.490678 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.490655    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:30.495044 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.495030    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:30.505186 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.505165    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 17:52:30.507235 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.507222    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-228.ec2.internal"
Jul 29 17:52:30.514755 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.514714    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 17:52:30.603372 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.603344    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:30.662799 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:30.662774    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:31.073212 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.073176    2580 apiserver.go:52] "Watching apiserver"
Jul 29 17:52:31.079937 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.079908    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 17:52:31.080458 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.080433    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-228.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt","openshift-cluster-node-tuning-operator/tuned-7ghgf","openshift-dns/node-resolver-7b96n","openshift-image-registry/node-ca-zh4v2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal","openshift-multus/multus-mw955","openshift-multus/network-metrics-daemon-rl5wd","kube-system/global-pull-secret-syncer-xmd4d","kube-system/konnectivity-agent-nf779","openshift-multus/multus-additional-cni-plugins-9t64h","openshift-network-diagnostics/network-check-target-wvwkg","openshift-network-operator/iptables-alerter-kxtqg","openshift-ovn-kubernetes/ovnkube-node-ph4nh"]
Jul 29 17:52:31.082472 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.082452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.084358 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.084337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kxtqg"
Jul 29 17:52:31.085715 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.085695    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 17:52:31.085940 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.085923    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 17:52:31.086621 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.086600    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 17:52:31.086836 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.086820    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.087380    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.087699    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-plb7c\""
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.089337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.089971    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7b96n"
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.090577    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.090811    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.091020    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zh4v2"
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.091954    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.092214    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9w5bh\""
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.092565    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.092874    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.093189    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.093356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:31.093942 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.093431    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:31.094745 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.094671    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.095931    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.096171    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tbtcr\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.096208    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.096480    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.096734    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.096744    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.096908    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.097214    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wjjxx\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.097256    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8svpb\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.097479    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.097492    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.097757    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.098997    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.099035    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-djbq2\""
Jul 29 17:52:31.099164 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.099087    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:31.100775 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.100035    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.100775 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.100265    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 17:52:31.109175 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109152    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b7f4a89-2914-4a7c-a5cd-69a9ab162439-host\") pod \"node-ca-zh4v2\" (UID: \"5b7f4a89-2914-4a7c-a5cd-69a9ab162439\") " pod="openshift-image-registry/node-ca-zh4v2"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-var-lib-openvswitch\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-node-log\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-ovnkube-script-lib\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-var-lib-kubelet\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109657    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b7f4a89-2914-4a7c-a5cd-69a9ab162439-serviceca\") pod \"node-ca-zh4v2\" (UID: \"5b7f4a89-2914-4a7c-a5cd-69a9ab162439\") " pod="openshift-image-registry/node-ca-zh4v2"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-log-socket\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109695    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9337a15-2a68-4c41-82c3-6cf81a81e1f6-tmp-dir\") pod \"node-resolver-7b96n\" (UID: \"f9337a15-2a68-4c41-82c3-6cf81a81e1f6\") " pod="openshift-dns/node-resolver-7b96n"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109702    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-cni-netd\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109737    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-run\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-tuned\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-etc-openvswitch\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-env-overrides\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h75gq\" (UniqueName: \"kubernetes.io/projected/8dc17568-9cb6-4605-bc11-14483b2533e6-kube-api-access-h75gq\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:31.112833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109885    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-var-lib-cni-bin\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-sysctl-conf\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9337a15-2a68-4c41-82c3-6cf81a81e1f6-hosts-file\") pod \"node-resolver-7b96n\" (UID: \"f9337a15-2a68-4c41-82c3-6cf81a81e1f6\") " pod="openshift-dns/node-resolver-7b96n"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.109981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h2jh\" (UniqueName: \"kubernetes.io/projected/5b7f4a89-2914-4a7c-a5cd-69a9ab162439-kube-api-access-8h2jh\") pod \"node-ca-zh4v2\" (UID: \"5b7f4a89-2914-4a7c-a5cd-69a9ab162439\") " pod="openshift-image-registry/node-ca-zh4v2"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.110008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-os-release\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.110057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-var-lib-cni-multus\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.110091    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-hostroot\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.110124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4ktx\" (UniqueName: \"kubernetes.io/projected/33d04ec1-2871-4af5-b149-f0ff5710cc22-kube-api-access-l4ktx\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.110150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/819b70d3-539f-493d-a313-ce06fdfb0871-cni-binary-copy\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.110184    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.110298    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.110630    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.110182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-lib-modules\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.110919    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7lc9b\""
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.110974    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-host\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-dbus\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-sysctl-d\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-cni-bin\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/819b70d3-539f-493d-a313-ce06fdfb0871-multus-daemon-config\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.113763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-run-netns\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111696    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-run-systemd\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-modprobe-d\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111804    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111828    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/721842a0-03b8-4521-b222-93d45f81dae8-iptables-alerter-script\") pod \"iptables-alerter-kxtqg\" (UID: \"721842a0-03b8-4521-b222-93d45f81dae8\") " pod="openshift-network-operator/iptables-alerter-kxtqg"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/721842a0-03b8-4521-b222-93d45f81dae8-host-slash\") pod \"iptables-alerter-kxtqg\" (UID: \"721842a0-03b8-4521-b222-93d45f81dae8\") " pod="openshift-network-operator/iptables-alerter-kxtqg"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-run-ovn\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.111973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-system-cni-dir\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-run-multus-certs\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvvgq\" (UniqueName: \"kubernetes.io/projected/721842a0-03b8-4521-b222-93d45f81dae8-kube-api-access-tvvgq\") pod \"iptables-alerter-kxtqg\" (UID: \"721842a0-03b8-4521-b222-93d45f81dae8\") " pod="openshift-network-operator/iptables-alerter-kxtqg"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-systemd-units\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-ovnkube-config\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112123    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-run-netns\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-etc-kubernetes\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112207    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-sys\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.114663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-var-lib-kubelet\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-sysconfig\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112291    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33d04ec1-2871-4af5-b149-f0ff5710cc22-tmp\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-kubelet-config\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-kubelet\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-kubernetes\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-multus-cni-dir\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112441    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-cnibin\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-multus-socket-dir-parent\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-multus-conf-dir\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112662    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-systemd\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-ovn-node-metrics-cert\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-run-k8s-cni-cncf-io\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpqf5\" (UniqueName: \"kubernetes.io/projected/819b70d3-539f-493d-a313-ce06fdfb0871-kube-api-access-rpqf5\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-run-openvswitch\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64gvm\" (UniqueName: \"kubernetes.io/projected/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-kube-api-access-64gvm\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.112953    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mtlv\" (UniqueName: \"kubernetes.io/projected/f9337a15-2a68-4c41-82c3-6cf81a81e1f6-kube-api-access-8mtlv\") pod \"node-resolver-7b96n\" (UID: \"f9337a15-2a68-4c41-82c3-6cf81a81e1f6\") " pod="openshift-dns/node-resolver-7b96n"
Jul 29 17:52:31.115560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.113074    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-slash\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.116410 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.113160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-run-ovn-kubernetes\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.116410 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.113639    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nhrjs\""
Jul 29 17:52:31.116410 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.113848    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 17:52:31.116410 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.114227    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qgz2n\""
Jul 29 17:52:31.116410 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.114957    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:31.116410 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.115067    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:31.116410 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.115226    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 17:52:31.118159 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.118094    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 17:52:31.137412 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.137388    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bc2cr"
Jul 29 17:52:31.146549 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.146517    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bc2cr"
Jul 29 17:52:31.196951 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.196928    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.213791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-run-ovn\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.213870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.213900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-system-cni-dir\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.213929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-run-multus-certs\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.213943    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-run-ovn\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.213955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvvgq\" (UniqueName: \"kubernetes.io/projected/721842a0-03b8-4521-b222-93d45f81dae8-kube-api-access-tvvgq\") pod \"iptables-alerter-kxtqg\" (UID: \"721842a0-03b8-4521-b222-93d45f81dae8\") " pod="openshift-network-operator/iptables-alerter-kxtqg"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-systemd-units\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-ovnkube-config\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214057    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-system-cni-dir\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-run-netns\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-run-multus-certs\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-etc-kubernetes\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-sys\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214118    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-systemd-units\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-var-lib-kubelet\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-sysconfig\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-run-netns\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.215236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33d04ec1-2871-4af5-b149-f0ff5710cc22-tmp\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.214207    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-kubelet-config\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-kubelet\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.214275    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs podName:8dc17568-9cb6-4605-bc11-14483b2533e6 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:31.71425163 +0000 UTC m=+2.983750414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs") pod "network-metrics-daemon-rl5wd" (UID: "8dc17568-9cb6-4605-bc11-14483b2533e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-kubelet\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-registration-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lt72\" (UniqueName: \"kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72\") pod \"network-check-target-wvwkg\" (UID: \"83cac0db-3ee2-4150-859f-1a0a1f720f66\") " pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-etc-kubernetes\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-sys\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214505    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-var-lib-kubelet\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-kubernetes\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-multus-cni-dir\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214570    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-kubelet-config\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-cnibin\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214592    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-kubernetes\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.216256 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214639    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-multus-socket-dir-parent\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214682    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-multus-socket-dir-parent\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-multus-conf-dir\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-systemd\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-ovn-node-metrics-cert\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-cnibin\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214758    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45bgs\" (UniqueName: \"kubernetes.io/projected/0ecdb051-5071-4fc8-a359-39a5c90733c2-kube-api-access-45bgs\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-multus-cni-dir\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-run-k8s-cni-cncf-io\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-systemd\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpqf5\" (UniqueName: \"kubernetes.io/projected/819b70d3-539f-493d-a313-ce06fdfb0871-kube-api-access-rpqf5\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214807    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-ovnkube-config\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-run-openvswitch\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64gvm\" (UniqueName: \"kubernetes.io/projected/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-kube-api-access-64gvm\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/19dd92a6-e163-466e-862e-0c79aebda97b-agent-certs\") pod \"konnectivity-agent-nf779\" (UID: \"19dd92a6-e163-466e-862e-0c79aebda97b\") " pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214886    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-multus-conf-dir\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/19dd92a6-e163-466e-862e-0c79aebda97b-konnectivity-ca\") pod \"konnectivity-agent-nf779\" (UID: \"19dd92a6-e163-466e-862e-0c79aebda97b\") " pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:52:31.217205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-run-openvswitch\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214953    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9e922cb-fba3-4b69-be76-b71586930fb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mtlv\" (UniqueName: \"kubernetes.io/projected/f9337a15-2a68-4c41-82c3-6cf81a81e1f6-kube-api-access-8mtlv\") pod \"node-resolver-7b96n\" (UID: \"f9337a15-2a68-4c41-82c3-6cf81a81e1f6\") " pod="openshift-dns/node-resolver-7b96n"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.214991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-slash\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-run-ovn-kubernetes\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215014    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-run-k8s-cni-cncf-io\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-socket-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b7f4a89-2914-4a7c-a5cd-69a9ab162439-host\") pod \"node-ca-zh4v2\" (UID: \"5b7f4a89-2914-4a7c-a5cd-69a9ab162439\") " pod="openshift-image-registry/node-ca-zh4v2"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215154    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-var-lib-openvswitch\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-node-log\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-ovnkube-script-lib\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-var-lib-kubelet\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215209    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b7f4a89-2914-4a7c-a5cd-69a9ab162439-host\") pod \"node-ca-zh4v2\" (UID: \"5b7f4a89-2914-4a7c-a5cd-69a9ab162439\") " pod="openshift-image-registry/node-ca-zh4v2"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b7f4a89-2914-4a7c-a5cd-69a9ab162439-serviceca\") pod \"node-ca-zh4v2\" (UID: \"5b7f4a89-2914-4a7c-a5cd-69a9ab162439\") " pod="openshift-image-registry/node-ca-zh4v2"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-slash\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-node-log\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.218066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-run-ovn-kubernetes\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-log-socket\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-sysconfig\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215284    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-var-lib-openvswitch\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.215307    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9337a15-2a68-4c41-82c3-6cf81a81e1f6-tmp-dir\") pod \"node-resolver-7b96n\" (UID: \"f9337a15-2a68-4c41-82c3-6cf81a81e1f6\") " pod="openshift-dns/node-resolver-7b96n"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-cni-netd\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215388    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-log-socket\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-var-lib-kubelet\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.215424    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret podName:45f70f28-b309-4c10-ad6b-39ebcc5abcfc nodeName:}" failed. No retries permitted until 2026-07-29 17:52:31.715406317 +0000 UTC m=+2.984905079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret") pod "global-pull-secret-syncer-xmd4d" (UID: "45f70f28-b309-4c10-ad6b-39ebcc5abcfc") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-run\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-tuned\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-etc-openvswitch\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-env-overrides\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h75gq\" (UniqueName: \"kubernetes.io/projected/8dc17568-9cb6-4605-bc11-14483b2533e6-kube-api-access-h75gq\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.219257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215686    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98lbk\" (UniqueName: \"kubernetes.io/projected/e9e922cb-fba3-4b69-be76-b71586930fb2-kube-api-access-98lbk\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-var-lib-cni-bin\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b7f4a89-2914-4a7c-a5cd-69a9ab162439-serviceca\") pod \"node-ca-zh4v2\" (UID: \"5b7f4a89-2914-4a7c-a5cd-69a9ab162439\") " pod="openshift-image-registry/node-ca-zh4v2"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-sysctl-conf\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215807    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-etc-openvswitch\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-run\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9337a15-2a68-4c41-82c3-6cf81a81e1f6-tmp-dir\") pod \"node-resolver-7b96n\" (UID: \"f9337a15-2a68-4c41-82c3-6cf81a81e1f6\") " pod="openshift-dns/node-resolver-7b96n"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.215972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-cni-netd\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216124    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-var-lib-cni-bin\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216210    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9337a15-2a68-4c41-82c3-6cf81a81e1f6-hosts-file\") pod \"node-resolver-7b96n\" (UID: \"f9337a15-2a68-4c41-82c3-6cf81a81e1f6\") " pod="openshift-dns/node-resolver-7b96n"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216231    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-sysctl-conf\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216250    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h2jh\" (UniqueName: \"kubernetes.io/projected/5b7f4a89-2914-4a7c-a5cd-69a9ab162439-kube-api-access-8h2jh\") pod \"node-ca-zh4v2\" (UID: \"5b7f4a89-2914-4a7c-a5cd-69a9ab162439\") " pod="openshift-image-registry/node-ca-zh4v2"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216278    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-os-release\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-var-lib-cni-multus\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-hostroot\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4ktx\" (UniqueName: \"kubernetes.io/projected/33d04ec1-2871-4af5-b149-f0ff5710cc22-kube-api-access-l4ktx\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/819b70d3-539f-493d-a313-ce06fdfb0871-cni-binary-copy\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.220134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-env-overrides\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216517    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-os-release\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216582    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f9337a15-2a68-4c41-82c3-6cf81a81e1f6-hosts-file\") pod \"node-resolver-7b96n\" (UID: \"f9337a15-2a68-4c41-82c3-6cf81a81e1f6\") " pod="openshift-dns/node-resolver-7b96n"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-ovnkube-script-lib\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-lib-modules\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216647    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-host-var-lib-cni-multus\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-host\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-dbus\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216730    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-lib-modules\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-device-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-host\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216793    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-sys-fs\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-system-cni-dir\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216864    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/819b70d3-539f-493d-a313-ce06fdfb0871-hostroot\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-sysctl-d\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-cni-bin\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.216972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-dbus\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-cni-bin\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.221068 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217055    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/819b70d3-539f-493d-a313-ce06fdfb0871-cni-binary-copy\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217072    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-sysctl-d\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9e922cb-fba3-4b69-be76-b71586930fb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/819b70d3-539f-493d-a313-ce06fdfb0871-multus-daemon-config\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-run-netns\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-run-netns\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-run-systemd\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-run-systemd\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217340    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-cnibin\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-os-release\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9e922cb-fba3-4b69-be76-b71586930fb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-modprobe-d\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/721842a0-03b8-4521-b222-93d45f81dae8-iptables-alerter-script\") pod \"iptables-alerter-kxtqg\" (UID: \"721842a0-03b8-4521-b222-93d45f81dae8\") " pod="openshift-network-operator/iptables-alerter-kxtqg"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/721842a0-03b8-4521-b222-93d45f81dae8-host-slash\") pod \"iptables-alerter-kxtqg\" (UID: \"721842a0-03b8-4521-b222-93d45f81dae8\") " pod="openshift-network-operator/iptables-alerter-kxtqg"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217539    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/819b70d3-539f-493d-a313-ce06fdfb0871-multus-daemon-config\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.221964 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217554    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-modprobe-d\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.222879 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.217591    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/721842a0-03b8-4521-b222-93d45f81dae8-host-slash\") pod \"iptables-alerter-kxtqg\" (UID: \"721842a0-03b8-4521-b222-93d45f81dae8\") " pod="openshift-network-operator/iptables-alerter-kxtqg"
Jul 29 17:52:31.222879 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.218108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/721842a0-03b8-4521-b222-93d45f81dae8-iptables-alerter-script\") pod \"iptables-alerter-kxtqg\" (UID: \"721842a0-03b8-4521-b222-93d45f81dae8\") " pod="openshift-network-operator/iptables-alerter-kxtqg"
Jul 29 17:52:31.222879 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.218443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33d04ec1-2871-4af5-b149-f0ff5710cc22-tmp\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.222879 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.218629    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-ovn-node-metrics-cert\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.222879 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.221058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/33d04ec1-2871-4af5-b149-f0ff5710cc22-etc-tuned\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.224906 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.224791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64gvm\" (UniqueName: \"kubernetes.io/projected/d29c60d8-0bbb-4be2-aa9f-647fc5d2833d-kube-api-access-64gvm\") pod \"ovnkube-node-ph4nh\" (UID: \"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d\") " pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.225421 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.225370    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpqf5\" (UniqueName: \"kubernetes.io/projected/819b70d3-539f-493d-a313-ce06fdfb0871-kube-api-access-rpqf5\") pod \"multus-mw955\" (UID: \"819b70d3-539f-493d-a313-ce06fdfb0871\") " pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.226676 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.226177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4ktx\" (UniqueName: \"kubernetes.io/projected/33d04ec1-2871-4af5-b149-f0ff5710cc22-kube-api-access-l4ktx\") pod \"tuned-7ghgf\" (UID: \"33d04ec1-2871-4af5-b149-f0ff5710cc22\") " pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.227620 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.227592    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvvgq\" (UniqueName: \"kubernetes.io/projected/721842a0-03b8-4521-b222-93d45f81dae8-kube-api-access-tvvgq\") pod \"iptables-alerter-kxtqg\" (UID: \"721842a0-03b8-4521-b222-93d45f81dae8\") " pod="openshift-network-operator/iptables-alerter-kxtqg"
Jul 29 17:52:31.228846 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.228798    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mtlv\" (UniqueName: \"kubernetes.io/projected/f9337a15-2a68-4c41-82c3-6cf81a81e1f6-kube-api-access-8mtlv\") pod \"node-resolver-7b96n\" (UID: \"f9337a15-2a68-4c41-82c3-6cf81a81e1f6\") " pod="openshift-dns/node-resolver-7b96n"
Jul 29 17:52:31.230025 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.229973    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h2jh\" (UniqueName: \"kubernetes.io/projected/5b7f4a89-2914-4a7c-a5cd-69a9ab162439-kube-api-access-8h2jh\") pod \"node-ca-zh4v2\" (UID: \"5b7f4a89-2914-4a7c-a5cd-69a9ab162439\") " pod="openshift-image-registry/node-ca-zh4v2"
Jul 29 17:52:31.230488 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.230466    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h75gq\" (UniqueName: \"kubernetes.io/projected/8dc17568-9cb6-4605-bc11-14483b2533e6-kube-api-access-h75gq\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:31.274513 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.274452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-228.ec2.internal" event={"ID":"d14432099af8323069e39bbc1aa22244","Type":"ContainerStarted","Data":"fb5bf88312f299b4e5f13b540f8d92ca5ecdc38691a560b817c3e65e9abd0f7a"}
Jul 29 17:52:31.276289 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.276259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal" event={"ID":"4d9f93c40a3798d4828fdeedd9a7ffca","Type":"ContainerStarted","Data":"7b45af92f944f03f80c352dc28ee110eb160b4e9200459fbdd8ce372d5889a44"}
Jul 29 17:52:31.318464 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-registration-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.318603 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lt72\" (UniqueName: \"kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72\") pod \"network-check-target-wvwkg\" (UID: \"83cac0db-3ee2-4150-859f-1a0a1f720f66\") " pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:31.318603 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45bgs\" (UniqueName: \"kubernetes.io/projected/0ecdb051-5071-4fc8-a359-39a5c90733c2-kube-api-access-45bgs\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.318603 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/19dd92a6-e163-466e-862e-0c79aebda97b-agent-certs\") pod \"konnectivity-agent-nf779\" (UID: \"19dd92a6-e163-466e-862e-0c79aebda97b\") " pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:52:31.318603 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/19dd92a6-e163-466e-862e-0c79aebda97b-konnectivity-ca\") pod \"konnectivity-agent-nf779\" (UID: \"19dd92a6-e163-466e-862e-0c79aebda97b\") " pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:52:31.318603 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9e922cb-fba3-4b69-be76-b71586930fb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.318880 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318580    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-registration-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.318880 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-socket-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.318880 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.318880 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.318880 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.318880 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98lbk\" (UniqueName: \"kubernetes.io/projected/e9e922cb-fba3-4b69-be76-b71586930fb2-kube-api-access-98lbk\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.319189 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-device-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.319189 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-sys-fs\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.319189 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-socket-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.319189 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-system-cni-dir\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.319189 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318933    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-etc-selinux\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.319189 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.318953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9e922cb-fba3-4b69-be76-b71586930fb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.319409 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-device-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.319409 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-sys-fs\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.319409 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319318    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/19dd92a6-e163-466e-862e-0c79aebda97b-konnectivity-ca\") pod \"konnectivity-agent-nf779\" (UID: \"19dd92a6-e163-466e-862e-0c79aebda97b\") " pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:52:31.319409 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319344    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-system-cni-dir\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.319530 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319376    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0ecdb051-5071-4fc8-a359-39a5c90733c2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.319592 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319575    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.319719 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-cnibin\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.319818 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-cnibin\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.319818 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319722    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9e922cb-fba3-4b69-be76-b71586930fb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.319818 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-os-release\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.319818 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9e922cb-fba3-4b69-be76-b71586930fb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.319818 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319796    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9e922cb-fba3-4b69-be76-b71586930fb2-os-release\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.320022 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.319879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9e922cb-fba3-4b69-be76-b71586930fb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.320412 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.320389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9e922cb-fba3-4b69-be76-b71586930fb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.321766 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.321742    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/19dd92a6-e163-466e-862e-0c79aebda97b-agent-certs\") pod \"konnectivity-agent-nf779\" (UID: \"19dd92a6-e163-466e-862e-0c79aebda97b\") " pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:52:31.324986 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.324943    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:31.324986 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.324974    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:31.324986 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.324990    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6lt72 for pod openshift-network-diagnostics/network-check-target-wvwkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:31.325291 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.325074    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72 podName:83cac0db-3ee2-4150-859f-1a0a1f720f66 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:31.825053949 +0000 UTC m=+3.094552742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6lt72" (UniqueName: "kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72") pod "network-check-target-wvwkg" (UID: "83cac0db-3ee2-4150-859f-1a0a1f720f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:31.327835 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.327811    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98lbk\" (UniqueName: \"kubernetes.io/projected/e9e922cb-fba3-4b69-be76-b71586930fb2-kube-api-access-98lbk\") pod \"multus-additional-cni-plugins-9t64h\" (UID: \"e9e922cb-fba3-4b69-be76-b71586930fb2\") " pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.327936 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.327909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45bgs\" (UniqueName: \"kubernetes.io/projected/0ecdb051-5071-4fc8-a359-39a5c90733c2-kube-api-access-45bgs\") pod \"aws-ebs-csi-driver-node-b9wvt\" (UID: \"0ecdb051-5071-4fc8-a359-39a5c90733c2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.418528 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.418451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mw955"
Jul 29 17:52:31.428913 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.428882    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kxtqg"
Jul 29 17:52:31.438248 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.438224    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7ghgf"
Jul 29 17:52:31.444766 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.444742    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7b96n"
Jul 29 17:52:31.457998 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.457977    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 17:52:31.459079 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.459059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zh4v2"
Jul 29 17:52:31.468193 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.468173    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:31.478235 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.478207    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt"
Jul 29 17:52:31.486335 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.486315    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:52:31.492437 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.492417    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9t64h"
Jul 29 17:52:31.722622 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.722543    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:31.722622 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.722595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:31.722793 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.722702    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:31.722793 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.722707    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:31.722793 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.722755    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret podName:45f70f28-b309-4c10-ad6b-39ebcc5abcfc nodeName:}" failed. No retries permitted until 2026-07-29 17:52:32.722740637 +0000 UTC m=+3.992239399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret") pod "global-pull-secret-syncer-xmd4d" (UID: "45f70f28-b309-4c10-ad6b-39ebcc5abcfc") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:31.722793 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.722769    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs podName:8dc17568-9cb6-4605-bc11-14483b2533e6 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:32.722761882 +0000 UTC m=+3.992260645 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs") pod "network-metrics-daemon-rl5wd" (UID: "8dc17568-9cb6-4605-bc11-14483b2533e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:31.925092 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:31.925044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lt72\" (UniqueName: \"kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72\") pod \"network-check-target-wvwkg\" (UID: \"83cac0db-3ee2-4150-859f-1a0a1f720f66\") " pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:31.925267 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.925238    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:31.925317 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.925267    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:31.925317 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.925282    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6lt72 for pod openshift-network-diagnostics/network-check-target-wvwkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:31.925398 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:31.925350    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72 podName:83cac0db-3ee2-4150-859f-1a0a1f720f66 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:32.925328846 +0000 UTC m=+4.194827634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6lt72" (UniqueName: "kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72") pod "network-check-target-wvwkg" (UID: "83cac0db-3ee2-4150-859f-1a0a1f720f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:32.149732 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:32.149638    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 17:47:31 +0000 UTC" deadline="2028-04-07 03:10:59.443468668 +0000 UTC"
Jul 29 17:52:32.149732 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:32.149680    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14817h18m27.293791749s"
Jul 29 17:52:32.410261 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:32.410181    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33d04ec1_2871_4af5_b149_f0ff5710cc22.slice/crio-c1f6e617affef8192b05f9cfae6be3cc4684652155222b20ffa7dcb8d15922df WatchSource:0}: Error finding container c1f6e617affef8192b05f9cfae6be3cc4684652155222b20ffa7dcb8d15922df: Status 404 returned error can't find the container with id c1f6e617affef8192b05f9cfae6be3cc4684652155222b20ffa7dcb8d15922df
Jul 29 17:52:32.412799 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:32.412725    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod819b70d3_539f_493d_a313_ce06fdfb0871.slice/crio-4ba8a8945e45c54ec0572661fd1dcc9f0479a3167c9a7abb2cf25f487078d081 WatchSource:0}: Error finding container 4ba8a8945e45c54ec0572661fd1dcc9f0479a3167c9a7abb2cf25f487078d081: Status 404 returned error can't find the container with id 4ba8a8945e45c54ec0572661fd1dcc9f0479a3167c9a7abb2cf25f487078d081
Jul 29 17:52:32.415962 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:32.415533    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd29c60d8_0bbb_4be2_aa9f_647fc5d2833d.slice/crio-8d09c6f11c34dca941717994ad511accc515d27662ac104c67c33ac2d804b688 WatchSource:0}: Error finding container 8d09c6f11c34dca941717994ad511accc515d27662ac104c67c33ac2d804b688: Status 404 returned error can't find the container with id 8d09c6f11c34dca941717994ad511accc515d27662ac104c67c33ac2d804b688
Jul 29 17:52:32.417427 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:32.417388    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9337a15_2a68_4c41_82c3_6cf81a81e1f6.slice/crio-8db5e2f48fc79a3c2c25147ab60df211a687396dd3c42be99352da47e484ae5b WatchSource:0}: Error finding container 8db5e2f48fc79a3c2c25147ab60df211a687396dd3c42be99352da47e484ae5b: Status 404 returned error can't find the container with id 8db5e2f48fc79a3c2c25147ab60df211a687396dd3c42be99352da47e484ae5b
Jul 29 17:52:32.419776 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:32.419756    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ecdb051_5071_4fc8_a359_39a5c90733c2.slice/crio-ec42560803c4824d7176f5025259baa8eb63cad4ff08823a8b7b85723223d43f WatchSource:0}: Error finding container ec42560803c4824d7176f5025259baa8eb63cad4ff08823a8b7b85723223d43f: Status 404 returned error can't find the container with id ec42560803c4824d7176f5025259baa8eb63cad4ff08823a8b7b85723223d43f
Jul 29 17:52:32.421566 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:32.420688    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e922cb_fba3_4b69_be76_b71586930fb2.slice/crio-b6493a586c72c5e50fadd931dd3fa751c1cfbc89dea32b3657a3bf937f5644d2 WatchSource:0}: Error finding container b6493a586c72c5e50fadd931dd3fa751c1cfbc89dea32b3657a3bf937f5644d2: Status 404 returned error can't find the container with id b6493a586c72c5e50fadd931dd3fa751c1cfbc89dea32b3657a3bf937f5644d2
Jul 29 17:52:32.421566 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:32.421304    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19dd92a6_e163_466e_862e_0c79aebda97b.slice/crio-f8a47314a9472069287da24954a60990702a9630762e0f88adddc097dec95ac8 WatchSource:0}: Error finding container f8a47314a9472069287da24954a60990702a9630762e0f88adddc097dec95ac8: Status 404 returned error can't find the container with id f8a47314a9472069287da24954a60990702a9630762e0f88adddc097dec95ac8
Jul 29 17:52:32.424786 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:32.424764    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7f4a89_2914_4a7c_a5cd_69a9ab162439.slice/crio-01ad521236d451c70a3844d7ca225b5bbabc498b47aa2ab80c9d0157579236b9 WatchSource:0}: Error finding container 01ad521236d451c70a3844d7ca225b5bbabc498b47aa2ab80c9d0157579236b9: Status 404 returned error can't find the container with id 01ad521236d451c70a3844d7ca225b5bbabc498b47aa2ab80c9d0157579236b9
Jul 29 17:52:32.427064 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:52:32.426808    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod721842a0_03b8_4521_b222_93d45f81dae8.slice/crio-55f9847543e13932df2bb0ce486b6832d62094e2758941dfe90e02dd2dad700b WatchSource:0}: Error finding container 55f9847543e13932df2bb0ce486b6832d62094e2758941dfe90e02dd2dad700b: Status 404 returned error can't find the container with id 55f9847543e13932df2bb0ce486b6832d62094e2758941dfe90e02dd2dad700b
Jul 29 17:52:32.730054 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:32.729877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:32.730171 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:32.730090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:32.730171 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:32.730033    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:32.730251 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:32.730177    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:32.730251 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:32.730184    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs podName:8dc17568-9cb6-4605-bc11-14483b2533e6 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:34.73016178 +0000 UTC m=+5.999660543 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs") pod "network-metrics-daemon-rl5wd" (UID: "8dc17568-9cb6-4605-bc11-14483b2533e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:32.730251 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:32.730214    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret podName:45f70f28-b309-4c10-ad6b-39ebcc5abcfc nodeName:}" failed. No retries permitted until 2026-07-29 17:52:34.730203231 +0000 UTC m=+5.999701994 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret") pod "global-pull-secret-syncer-xmd4d" (UID: "45f70f28-b309-4c10-ad6b-39ebcc5abcfc") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:32.932693 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:32.932565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lt72\" (UniqueName: \"kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72\") pod \"network-check-target-wvwkg\" (UID: \"83cac0db-3ee2-4150-859f-1a0a1f720f66\") " pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:32.932848 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:32.932709    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:32.932848 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:32.932733    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:32.932848 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:32.932742    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6lt72 for pod openshift-network-diagnostics/network-check-target-wvwkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:32.932848 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:32.932807    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72 podName:83cac0db-3ee2-4150-859f-1a0a1f720f66 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:34.93278722 +0000 UTC m=+6.202285987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6lt72" (UniqueName: "kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72") pod "network-check-target-wvwkg" (UID: "83cac0db-3ee2-4150-859f-1a0a1f720f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:33.150251 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.150183    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 17:47:31 +0000 UTC" deadline="2028-01-13 22:34:33.185305519 +0000 UTC"
Jul 29 17:52:33.150251 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.150250    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12796h42m0.035059901s"
Jul 29 17:52:33.268949 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.268874    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:33.269119 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:33.269002    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:33.269119 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.269111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:33.269256 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:33.269191    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:33.269315 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.269256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:33.269371 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:33.269346    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:33.290805 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.290155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-228.ec2.internal" event={"ID":"d14432099af8323069e39bbc1aa22244","Type":"ContainerStarted","Data":"32608447c8d2170bb6bf1eebc7c9464339161a385fa901581dac9c81d9b5ed40"}
Jul 29 17:52:33.297354 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.297326    2580 generic.go:356] "Generic (PLEG): container finished" podID="4d9f93c40a3798d4828fdeedd9a7ffca" containerID="0411590a4e7048b5705df315c1fda31a37b451e0fe065bb5f24ae57ba17ebc96" exitCode=0
Jul 29 17:52:33.297470 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.297417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal" event={"ID":"4d9f93c40a3798d4828fdeedd9a7ffca","Type":"ContainerDied","Data":"0411590a4e7048b5705df315c1fda31a37b451e0fe065bb5f24ae57ba17ebc96"}
Jul 29 17:52:33.308912 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.308884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nf779" event={"ID":"19dd92a6-e163-466e-862e-0c79aebda97b","Type":"ContainerStarted","Data":"f8a47314a9472069287da24954a60990702a9630762e0f88adddc097dec95ac8"}
Jul 29 17:52:33.318802 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.318770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9t64h" event={"ID":"e9e922cb-fba3-4b69-be76-b71586930fb2","Type":"ContainerStarted","Data":"b6493a586c72c5e50fadd931dd3fa751c1cfbc89dea32b3657a3bf937f5644d2"}
Jul 29 17:52:33.320823 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.320775    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-228.ec2.internal" podStartSLOduration=3.320757962 podStartE2EDuration="3.320757962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:33.304090686 +0000 UTC m=+4.573589475" watchObservedRunningTime="2026-07-29 17:52:33.320757962 +0000 UTC m=+4.590256751"
Jul 29 17:52:33.338755 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.336034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt" event={"ID":"0ecdb051-5071-4fc8-a359-39a5c90733c2","Type":"ContainerStarted","Data":"ec42560803c4824d7176f5025259baa8eb63cad4ff08823a8b7b85723223d43f"}
Jul 29 17:52:33.342243 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.342051    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7b96n" event={"ID":"f9337a15-2a68-4c41-82c3-6cf81a81e1f6","Type":"ContainerStarted","Data":"8db5e2f48fc79a3c2c25147ab60df211a687396dd3c42be99352da47e484ae5b"}
Jul 29 17:52:33.353703 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.353674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mw955" event={"ID":"819b70d3-539f-493d-a313-ce06fdfb0871","Type":"ContainerStarted","Data":"4ba8a8945e45c54ec0572661fd1dcc9f0479a3167c9a7abb2cf25f487078d081"}
Jul 29 17:52:33.362195 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.362139    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7ghgf" event={"ID":"33d04ec1-2871-4af5-b149-f0ff5710cc22","Type":"ContainerStarted","Data":"c1f6e617affef8192b05f9cfae6be3cc4684652155222b20ffa7dcb8d15922df"}
Jul 29 17:52:33.380440 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.380410    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kxtqg" event={"ID":"721842a0-03b8-4521-b222-93d45f81dae8","Type":"ContainerStarted","Data":"55f9847543e13932df2bb0ce486b6832d62094e2758941dfe90e02dd2dad700b"}
Jul 29 17:52:33.388917 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.388888    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zh4v2" event={"ID":"5b7f4a89-2914-4a7c-a5cd-69a9ab162439","Type":"ContainerStarted","Data":"01ad521236d451c70a3844d7ca225b5bbabc498b47aa2ab80c9d0157579236b9"}
Jul 29 17:52:33.405111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:33.405082    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh" event={"ID":"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d","Type":"ContainerStarted","Data":"8d09c6f11c34dca941717994ad511accc515d27662ac104c67c33ac2d804b688"}
Jul 29 17:52:34.426720 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:34.426582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal" event={"ID":"4d9f93c40a3798d4828fdeedd9a7ffca","Type":"ContainerStarted","Data":"47b52904a7f97216b46585f665f099ce2fa7fe6b697d7cdfdaef96caf1a1cdd2"}
Jul 29 17:52:34.441247 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:34.440110    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-228.ec2.internal" podStartSLOduration=4.440090678 podStartE2EDuration="4.440090678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:34.43976471 +0000 UTC m=+5.709263498" watchObservedRunningTime="2026-07-29 17:52:34.440090678 +0000 UTC m=+5.709589467"
Jul 29 17:52:34.755417 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:34.755330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:34.755417 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:34.755404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:34.755650 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:34.755565    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:34.755650 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:34.755628    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs podName:8dc17568-9cb6-4605-bc11-14483b2533e6 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:38.755607159 +0000 UTC m=+10.025105928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs") pod "network-metrics-daemon-rl5wd" (UID: "8dc17568-9cb6-4605-bc11-14483b2533e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:34.756212 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:34.756187    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:34.756322 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:34.756251    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret podName:45f70f28-b309-4c10-ad6b-39ebcc5abcfc nodeName:}" failed. No retries permitted until 2026-07-29 17:52:38.75623411 +0000 UTC m=+10.025732880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret") pod "global-pull-secret-syncer-xmd4d" (UID: "45f70f28-b309-4c10-ad6b-39ebcc5abcfc") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:34.957799 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:34.957762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lt72\" (UniqueName: \"kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72\") pod \"network-check-target-wvwkg\" (UID: \"83cac0db-3ee2-4150-859f-1a0a1f720f66\") " pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:34.957973 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:34.957947    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:34.957973 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:34.957967    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:34.958112 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:34.957982    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6lt72 for pod openshift-network-diagnostics/network-check-target-wvwkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:34.958112 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:34.958057    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72 podName:83cac0db-3ee2-4150-859f-1a0a1f720f66 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:38.958035829 +0000 UTC m=+10.227534594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6lt72" (UniqueName: "kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72") pod "network-check-target-wvwkg" (UID: "83cac0db-3ee2-4150-859f-1a0a1f720f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:35.268907 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:35.268878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:35.269095 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:35.269006    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:35.269476 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:35.269455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:35.269589 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:35.269568    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:35.269667 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:35.269628    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:35.269728 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:35.269700    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:37.268403 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:37.268370    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:37.268868 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:37.268476    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:37.268868 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:37.268763    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:37.268868 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:37.268826    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:37.269059 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:37.268873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:37.269059 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:37.268913    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:38.788181 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:38.788140    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:38.788579 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:38.788220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:38.788579 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:38.788339    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:38.788579 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:38.788456    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs podName:8dc17568-9cb6-4605-bc11-14483b2533e6 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:46.788434747 +0000 UTC m=+18.057933510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs") pod "network-metrics-daemon-rl5wd" (UID: "8dc17568-9cb6-4605-bc11-14483b2533e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:38.788867 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:38.788848    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:38.788949 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:38.788902    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret podName:45f70f28-b309-4c10-ad6b-39ebcc5abcfc nodeName:}" failed. No retries permitted until 2026-07-29 17:52:46.788887424 +0000 UTC m=+18.058386187 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret") pod "global-pull-secret-syncer-xmd4d" (UID: "45f70f28-b309-4c10-ad6b-39ebcc5abcfc") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:38.989701 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:38.989623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lt72\" (UniqueName: \"kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72\") pod \"network-check-target-wvwkg\" (UID: \"83cac0db-3ee2-4150-859f-1a0a1f720f66\") " pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:38.989886 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:38.989870    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:38.989977 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:38.989891    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:38.989977 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:38.989901    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6lt72 for pod openshift-network-diagnostics/network-check-target-wvwkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:38.990066 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:38.989990    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72 podName:83cac0db-3ee2-4150-859f-1a0a1f720f66 nodeName:}" failed. No retries permitted until 2026-07-29 17:52:46.989936376 +0000 UTC m=+18.259435140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6lt72" (UniqueName: "kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72") pod "network-check-target-wvwkg" (UID: "83cac0db-3ee2-4150-859f-1a0a1f720f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:39.267908 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:39.267825    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:39.268086 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:39.267935    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:39.268569 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:39.268294    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:39.268569 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:39.268430    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:39.268569 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:39.268478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:39.268569 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:39.268538    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:41.268374 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:41.268343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:41.268820 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:41.268391    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:41.268820 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:41.268465    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:41.268820 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:41.268514    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:41.268820 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:41.268603    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:41.268820 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:41.268687    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:43.266994 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:43.266945    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:43.266994 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:43.266990    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:43.267499 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:43.266944    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:43.267499 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:43.267097    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:43.267499 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:43.267151    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:43.267499 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:43.267216    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:45.267531 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:45.267075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:45.267531 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:45.267095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:45.267531 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:45.267195    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:45.267531 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:45.267249    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:45.267531 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:45.267361    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:45.267531 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:45.267435    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:46.857089 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:46.857052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:46.857585 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:46.857115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:46.857585 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:46.857204    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:46.857585 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:46.857204    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:46.857585 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:46.857267    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs podName:8dc17568-9cb6-4605-bc11-14483b2533e6 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:02.857252119 +0000 UTC m=+34.126750883 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs") pod "network-metrics-daemon-rl5wd" (UID: "8dc17568-9cb6-4605-bc11-14483b2533e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:52:46.857585 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:46.857281    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret podName:45f70f28-b309-4c10-ad6b-39ebcc5abcfc nodeName:}" failed. No retries permitted until 2026-07-29 17:53:02.857275761 +0000 UTC m=+34.126774525 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret") pod "global-pull-secret-syncer-xmd4d" (UID: "45f70f28-b309-4c10-ad6b-39ebcc5abcfc") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:52:47.058739 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:47.058704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lt72\" (UniqueName: \"kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72\") pod \"network-check-target-wvwkg\" (UID: \"83cac0db-3ee2-4150-859f-1a0a1f720f66\") " pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:47.058923 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:47.058866    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:52:47.058923 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:47.058890    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:52:47.058923 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:47.058902    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6lt72 for pod openshift-network-diagnostics/network-check-target-wvwkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:47.059113 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:47.058973    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72 podName:83cac0db-3ee2-4150-859f-1a0a1f720f66 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:03.058943119 +0000 UTC m=+34.328441879 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6lt72" (UniqueName: "kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72") pod "network-check-target-wvwkg" (UID: "83cac0db-3ee2-4150-859f-1a0a1f720f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:52:47.266601 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:47.266520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:47.266601 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:47.266529    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:47.266818 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:47.266651    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:47.266940 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:47.266901    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:47.267085 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:47.267010    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:47.267085 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:47.267073    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:49.267968 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:49.267534    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:49.267968 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:49.267616    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:49.267968 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:49.267628    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:49.267968 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:49.267713    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:49.267968 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:49.267779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:49.267968 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:49.267846    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:51.266924 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:51.266893    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:51.267400 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:51.267053    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:51.267400 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:51.267083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:51.267400 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:51.267107    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:51.267400 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:51.267189    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:51.267400 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:51.267252    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:52.466733 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.466328    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9e922cb-fba3-4b69-be76-b71586930fb2" containerID="244f70798192728517ee080fa7bd72c126996f5ac4d39db61165e38e0043a741" exitCode=0
Jul 29 17:52:52.467622 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.466422    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9t64h" event={"ID":"e9e922cb-fba3-4b69-be76-b71586930fb2","Type":"ContainerDied","Data":"244f70798192728517ee080fa7bd72c126996f5ac4d39db61165e38e0043a741"}
Jul 29 17:52:52.469497 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.469465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt" event={"ID":"0ecdb051-5071-4fc8-a359-39a5c90733c2","Type":"ContainerStarted","Data":"11003fde5f391498440ddbedb879ca7e6a355ae2cfc52b9cd83dc0eb86c70b10"}
Jul 29 17:52:52.471362 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.471336    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7b96n" event={"ID":"f9337a15-2a68-4c41-82c3-6cf81a81e1f6","Type":"ContainerStarted","Data":"54cc29b5b04c4e37e35b4ea59b71a12fa71c69c82049085367d5d3730fca4cf7"}
Jul 29 17:52:52.472653 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.472632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mw955" event={"ID":"819b70d3-539f-493d-a313-ce06fdfb0871","Type":"ContainerStarted","Data":"b34ba88a393af622fd3db7dd9808afcef03d74462cf4ecce7137ac4bcb012572"}
Jul 29 17:52:52.477591 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.477566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7ghgf" event={"ID":"33d04ec1-2871-4af5-b149-f0ff5710cc22","Type":"ContainerStarted","Data":"103d2b1e9faf4ceb6590ed4bffc4fc4f2f6a3d6c4e38b758a442c683d5fd84e1"}
Jul 29 17:52:52.479009 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.478983    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zh4v2" event={"ID":"5b7f4a89-2914-4a7c-a5cd-69a9ab162439","Type":"ContainerStarted","Data":"0db0c11128f3ca07bf318088c9e6d7ccde2f53df0b11e100781079bb20ba1695"}
Jul 29 17:52:52.482225 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.482199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh" event={"ID":"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d","Type":"ContainerStarted","Data":"83bf3a05968df029a6d37013a476e0abc9f99b6522ffe0e95f834a9a13c75e82"}
Jul 29 17:52:52.482337 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.482259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh" event={"ID":"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d","Type":"ContainerStarted","Data":"9425a7da074b5ee7f5f4c3967bd8fb768ecc07c94e642e9a323e30bdcf6b2421"}
Jul 29 17:52:52.482337 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.482308    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh" event={"ID":"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d","Type":"ContainerStarted","Data":"53bc60dece6fecd8e5063d3c80f0cf05e037b8395f96629d29f5756f86db4175"}
Jul 29 17:52:52.482439 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.482344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh" event={"ID":"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d","Type":"ContainerStarted","Data":"f9598ea17927b78fd281c87d788ded5cdfd100dd49a98c8ab18caf5221027e83"}
Jul 29 17:52:52.482439 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.482382    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh" event={"ID":"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d","Type":"ContainerStarted","Data":"6a1ccd7e04c4c6e59adfc0c39535143acfd0a586a4560e14b811a7c3989553ab"}
Jul 29 17:52:52.482439 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.482416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh" event={"ID":"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d","Type":"ContainerStarted","Data":"2e162f8ac3886a31e64f7446873969aea6079371008ddbb7e77d085aeec2cef9"}
Jul 29 17:52:52.483634 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.483603    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nf779" event={"ID":"19dd92a6-e163-466e-862e-0c79aebda97b","Type":"ContainerStarted","Data":"c65d89c2fc8bc14634420b3f77f6fc44464db488ede625a5f7ba77238ca17f89"}
Jul 29 17:52:52.629713 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.629673    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nf779" podStartSLOduration=12.097742322 podStartE2EDuration="23.629657263s" totalImagesPullingTime="11.531914941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:52.578720738 +0000 UTC m=+23.848219521" watchObservedRunningTime="2026-07-29 17:52:52.629657263 +0000 UTC m=+23.899156108"
Jul 29 17:52:52.630099 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.630071    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7ghgf" podStartSLOduration=4.418279721 podStartE2EDuration="23.630061901s" totalImagesPullingTime="19.21178218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:52.629424677 +0000 UTC m=+23.898923490" watchObservedRunningTime="2026-07-29 17:52:52.630061901 +0000 UTC m=+23.899560689"
Jul 29 17:52:52.680040 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.680007    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7b96n" podStartSLOduration=4.502584305 podStartE2EDuration="23.679993329s" totalImagesPullingTime="19.177409024s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:52.679908592 +0000 UTC m=+23.949407375" watchObservedRunningTime="2026-07-29 17:52:52.679993329 +0000 UTC m=+23.949492111"
Jul 29 17:52:52.718162 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:52.718099    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mw955" podStartSLOduration=4.483858906 podStartE2EDuration="23.718086533s" totalImagesPullingTime="19.234227627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:52.717624042 +0000 UTC m=+23.987122825" watchObservedRunningTime="2026-07-29 17:52:52.718086533 +0000 UTC m=+23.987585315"
Jul 29 17:52:53.016197 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.016167    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zh4v2_5b7f4a89-2914-4a7c-a5cd-69a9ab162439/node-ca/0.log"
Jul 29 17:52:53.088087 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.088057    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 17:52:53.159605 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.159496    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T17:52:53.088082773Z","UUID":"8fb933d5-72d3-4139-9b70-b498226370c6","Handler":null,"Name":"","Endpoint":""}
Jul 29 17:52:53.161388 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.161365    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 17:52:53.161518 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.161405    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 17:52:53.266475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.266403    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:53.266475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.266425    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:53.266821 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.266416    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:53.266821 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:53.266530    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:53.266821 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:53.266622    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:53.266821 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:53.266695    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:53.488336 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.488294    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt" event={"ID":"0ecdb051-5071-4fc8-a359-39a5c90733c2","Type":"ContainerStarted","Data":"7b1c81dcf0930a9943a8aa32565d6082ab15a16546fe26760558e9bb8cabf96b"}
Jul 29 17:52:53.490036 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.490003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kxtqg" event={"ID":"721842a0-03b8-4521-b222-93d45f81dae8","Type":"ContainerStarted","Data":"2450e65836dcb1d0e8f7b97c55a731557694b341cae5cd44721e98ba1393f1bc"}
Jul 29 17:52:53.513121 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.513078    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zh4v2" podStartSLOduration=5.345462726 podStartE2EDuration="24.51306436s" totalImagesPullingTime="19.167601634s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:52.749433039 +0000 UTC m=+24.018931832" watchObservedRunningTime="2026-07-29 17:52:53.51306436 +0000 UTC m=+24.782563184"
Jul 29 17:52:53.513473 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:53.513444    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kxtqg" podStartSLOduration=5.347854762 podStartE2EDuration="24.51343627s" totalImagesPullingTime="19.165581508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:53.513032693 +0000 UTC m=+24.782531477" watchObservedRunningTime="2026-07-29 17:52:53.51343627 +0000 UTC m=+24.782935052"
Jul 29 17:52:54.497458 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:54.497416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh" event={"ID":"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d","Type":"ContainerStarted","Data":"40589cdd2b0246d4fff6d83592027f02f3b328c69edc490c469525e8ad56a692"}
Jul 29 17:52:55.267132 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:55.266933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:55.267388 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:55.267213    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:55.267388 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:55.266933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:55.267388 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:55.267279    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:55.267388 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:55.266933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:55.267388 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:55.267361    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:55.502534 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:55.502488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt" event={"ID":"0ecdb051-5071-4fc8-a359-39a5c90733c2","Type":"ContainerStarted","Data":"7b4e7c6e44a4783753bdf2a07aa69b7d94051f8f146ce67fd8f13571e2d03893"}
Jul 29 17:52:55.521487 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:55.521383    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b9wvt" podStartSLOduration=4.7020230640000005 podStartE2EDuration="26.52136804s" totalImagesPullingTime="21.819344976s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:55.520905748 +0000 UTC m=+26.790404531" watchObservedRunningTime="2026-07-29 17:52:55.52136804 +0000 UTC m=+26.790866823"
Jul 29 17:52:57.038870 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.038612    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:52:57.039628 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.039250    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:52:57.266771 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.266666    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:57.266919 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:57.266773    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:57.266919 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.266848    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:57.267019 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:57.266975    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:57.267104 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.267029    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:57.267143 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:57.267119    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:57.507821 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.507790    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9e922cb-fba3-4b69-be76-b71586930fb2" containerID="af974a5afd108bfb206b62b25f558cbbba2e6eac1d6c7a31e019386a3458b471" exitCode=0
Jul 29 17:52:57.508003 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.507859    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9t64h" event={"ID":"e9e922cb-fba3-4b69-be76-b71586930fb2","Type":"ContainerDied","Data":"af974a5afd108bfb206b62b25f558cbbba2e6eac1d6c7a31e019386a3458b471"}
Jul 29 17:52:57.511693 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.511674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh" event={"ID":"d29c60d8-0bbb-4be2-aa9f-647fc5d2833d","Type":"ContainerStarted","Data":"ebf79c845ab79507af2506af61a31960fd5d6068520ba4a9ab26499f49fdf2b7"}
Jul 29 17:52:57.512015 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.511988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:57.512134 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.512068    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:57.515652 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.515542    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:57.530303 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.530251    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:57.530396 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.530339    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:52:57.598705 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:57.598657    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh" podStartSLOduration=9.355627558 podStartE2EDuration="28.59864173s" totalImagesPullingTime="19.243014172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:52:57.598502403 +0000 UTC m=+28.868001186" watchObservedRunningTime="2026-07-29 17:52:57.59864173 +0000 UTC m=+28.868140515"
Jul 29 17:52:59.267231 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:59.267209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:52:59.267611 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:59.267288    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:52:59.267611 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:59.267341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:52:59.267611 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:59.267414    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:52:59.267611 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:59.267458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:52:59.267611 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:52:59.267554    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:52:59.517412 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:59.517325    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9e922cb-fba3-4b69-be76-b71586930fb2" containerID="4e9e12ad4bb70db7c170b1fbf355e7209c52b578d3455a38857e6f635ad66e17" exitCode=0
Jul 29 17:52:59.517567 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:59.517412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9t64h" event={"ID":"e9e922cb-fba3-4b69-be76-b71586930fb2","Type":"ContainerDied","Data":"4e9e12ad4bb70db7c170b1fbf355e7209c52b578d3455a38857e6f635ad66e17"}
Jul 29 17:52:59.542660 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:59.542635    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:52:59.542769 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:59.542744    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 29 17:52:59.543167 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:52:59.543135    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nf779"
Jul 29 17:53:01.266731 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:01.266702    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:01.266731 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:01.266723    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:01.267280 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:01.266722    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:01.267280 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:01.266809    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:01.267280 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:01.266900    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:01.267280 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:01.267007    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:01.523108 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:01.523023    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9e922cb-fba3-4b69-be76-b71586930fb2" containerID="ecdad6b3c63f4d961dc4d05f67b07556f6386d9191213e790926382cf293d799" exitCode=0
Jul 29 17:53:01.523108 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:01.523081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9t64h" event={"ID":"e9e922cb-fba3-4b69-be76-b71586930fb2","Type":"ContainerDied","Data":"ecdad6b3c63f4d961dc4d05f67b07556f6386d9191213e790926382cf293d799"}
Jul 29 17:53:02.879091 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:02.879055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:02.879460 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:02.879128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:02.879460 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:02.879225    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:02.879460 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:02.879247    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:02.879460 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:02.879304    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret podName:45f70f28-b309-4c10-ad6b-39ebcc5abcfc nodeName:}" failed. No retries permitted until 2026-07-29 17:53:34.879282623 +0000 UTC m=+66.148781401 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret") pod "global-pull-secret-syncer-xmd4d" (UID: "45f70f28-b309-4c10-ad6b-39ebcc5abcfc") : object "kube-system"/"original-pull-secret" not registered
Jul 29 17:53:02.879460 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:02.879324    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs podName:8dc17568-9cb6-4605-bc11-14483b2533e6 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:34.879314055 +0000 UTC m=+66.148812822 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs") pod "network-metrics-daemon-rl5wd" (UID: "8dc17568-9cb6-4605-bc11-14483b2533e6") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 17:53:03.080440 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:03.080407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lt72\" (UniqueName: \"kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72\") pod \"network-check-target-wvwkg\" (UID: \"83cac0db-3ee2-4150-859f-1a0a1f720f66\") " pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:03.080613 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:03.080560    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 17:53:03.080613 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:03.080583    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 17:53:03.080613 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:03.080595    2580 projected.go:196] Error preparing data for projected volume kube-api-access-6lt72 for pod openshift-network-diagnostics/network-check-target-wvwkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:03.080774 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:03.080655    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72 podName:83cac0db-3ee2-4150-859f-1a0a1f720f66 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:35.080636295 +0000 UTC m=+66.350135055 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6lt72" (UniqueName: "kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72") pod "network-check-target-wvwkg" (UID: "83cac0db-3ee2-4150-859f-1a0a1f720f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 17:53:03.266325 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:03.266246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:03.266486 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:03.266247    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:03.266486 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:03.266366    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:03.266486 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:03.266248    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:03.266486 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:03.266457    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:03.266660 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:03.266529    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:05.266611 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:05.266579    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:05.267120 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:05.266580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:05.267120 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:05.266699    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:05.267120 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:05.266720    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:05.267120 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:05.266815    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:05.267120 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:05.266938    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:06.536847 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:06.536810    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9e922cb-fba3-4b69-be76-b71586930fb2" containerID="7da2131567db60ebd2368192282afe723b20c0955d3f3f72bc70d136b1cd2300" exitCode=0
Jul 29 17:53:06.537248 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:06.536860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9t64h" event={"ID":"e9e922cb-fba3-4b69-be76-b71586930fb2","Type":"ContainerDied","Data":"7da2131567db60ebd2368192282afe723b20c0955d3f3f72bc70d136b1cd2300"}
Jul 29 17:53:07.267051 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:07.267015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:07.267051 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:07.267033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:07.267270 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:07.267148    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:07.267270 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:07.267151    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:07.267270 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:07.267223    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:07.267378 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:07.267303    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:07.542308 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:07.542280    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9e922cb-fba3-4b69-be76-b71586930fb2" containerID="82462999b0b7220872b00b4025a3bcd1d3d22466619b8459ee5b3ca66a03d9fa" exitCode=0
Jul 29 17:53:07.542654 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:07.542329    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9t64h" event={"ID":"e9e922cb-fba3-4b69-be76-b71586930fb2","Type":"ContainerDied","Data":"82462999b0b7220872b00b4025a3bcd1d3d22466619b8459ee5b3ca66a03d9fa"}
Jul 29 17:53:08.547164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:08.547131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9t64h" event={"ID":"e9e922cb-fba3-4b69-be76-b71586930fb2","Type":"ContainerStarted","Data":"08070027863a51ef2e665f50dddaab5c5e3e1986f3c7d2761da186ddd883d3f9"}
Jul 29 17:53:08.570303 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:08.570258    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9t64h" podStartSLOduration=9.879550856 podStartE2EDuration="39.570244648s" totalImagesPullingTime="29.690693792s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:08.569246299 +0000 UTC m=+39.838745082" watchObservedRunningTime="2026-07-29 17:53:08.570244648 +0000 UTC m=+39.839743431"
Jul 29 17:53:09.266933 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:09.266904    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:09.267130 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:09.267009    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:09.267130 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:09.267051    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:09.267218 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:09.267052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:09.267218 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:09.267155    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:09.267287 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:09.267221    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:11.266905 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:11.266873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:11.267409 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:11.266872    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:11.267409 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:11.266992    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:11.267409 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:11.266994    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:11.267409 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:11.267068    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:11.267409 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:11.267120    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:13.266591 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:13.266560    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:13.267049 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:13.266560    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:13.267049 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:13.266668    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:13.267049 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:13.266562    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:13.267049 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:13.266737    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:13.267049 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:13.266833    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:15.266076 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:15.266044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:15.266528 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:15.266046    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:15.266528 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:15.266153    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:15.266528 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:15.266237    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:15.266528 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:15.266284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:15.266528 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:15.266330    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:17.266855 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:17.266686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:17.267298 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:17.266693    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:17.267298 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:17.266968    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:17.267298 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:17.266719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:17.267298 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:17.267023    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:17.267298 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:17.267093    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:17.774986 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:17.774906    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xmd4d"]
Jul 29 17:53:17.775146 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:17.775030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:17.775146 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:17.775129    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:17.776043 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:17.776018    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wvwkg"]
Jul 29 17:53:17.776698 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:17.776409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:17.776698 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:17.776525    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:17.776698 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:17.776632    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rl5wd"]
Jul 29 17:53:17.776862 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:17.776709    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:17.776862 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:17.776802    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:19.266981 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:19.266892    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:19.267641 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:19.266992    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:19.267641 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:19.267040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:19.267641 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:19.267159    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:19.267641 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:19.267201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:19.267641 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:19.267258    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:21.269131 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:21.269101    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:21.269521 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:21.269198    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xmd4d" podUID="45f70f28-b309-4c10-ad6b-39ebcc5abcfc"
Jul 29 17:53:21.269521 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:21.269208    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:21.269521 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:21.269304    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rl5wd" podUID="8dc17568-9cb6-4605-bc11-14483b2533e6"
Jul 29 17:53:21.269521 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:21.269319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:21.269521 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:21.269362    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wvwkg" podUID="83cac0db-3ee2-4150-859f-1a0a1f720f66"
Jul 29 17:53:22.520432 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.520365    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-228.ec2.internal" event="NodeReady"
Jul 29 17:53:22.520791 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.520489    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 17:53:22.570800 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.570771    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8mqfn"]
Jul 29 17:53:22.595015 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.594983    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9nhcg"]
Jul 29 17:53:22.595191 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.595047    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8mqfn"
Jul 29 17:53:22.597307 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.597279    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 17:53:22.597553 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.597533    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 17:53:22.597706 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.597536    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 17:53:22.598795 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.598776    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nmb94\""
Jul 29 17:53:22.618890 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.618862    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8mqfn"]
Jul 29 17:53:22.619008 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.618897    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9nhcg"]
Jul 29 17:53:22.619075 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.619031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.621357 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.621339    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fvz84\""
Jul 29 17:53:22.621357 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.621352    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 17:53:22.621693 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.621678    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 17:53:22.621769 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.621733    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 17:53:22.623205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.623190    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 17:53:22.627770 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.627752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/066e69d4-22ad-4231-b7c9-4b55475efeee-cert\") pod \"ingress-canary-8mqfn\" (UID: \"066e69d4-22ad-4231-b7c9-4b55475efeee\") " pod="openshift-ingress-canary/ingress-canary-8mqfn"
Jul 29 17:53:22.627845 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.627782    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5qcr\" (UniqueName: \"kubernetes.io/projected/066e69d4-22ad-4231-b7c9-4b55475efeee-kube-api-access-d5qcr\") pod \"ingress-canary-8mqfn\" (UID: \"066e69d4-22ad-4231-b7c9-4b55475efeee\") " pod="openshift-ingress-canary/ingress-canary-8mqfn"
Jul 29 17:53:22.669829 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.669797    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v8gbz"]
Jul 29 17:53:22.690845 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.690819    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v8gbz"]
Jul 29 17:53:22.691006 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.690938    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.694975 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.694942    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 17:53:22.694975 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.694946    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 17:53:22.695422 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.695405    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tr8hk\""
Jul 29 17:53:22.728585 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.728553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bedfbcbc-a007-4186-9405-989cdc0afd42-metrics-tls\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.728714 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.728586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5e5a0c7c-c535-4652-968b-f177289a242e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.728714 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.728628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/066e69d4-22ad-4231-b7c9-4b55475efeee-cert\") pod \"ingress-canary-8mqfn\" (UID: \"066e69d4-22ad-4231-b7c9-4b55475efeee\") " pod="openshift-ingress-canary/ingress-canary-8mqfn"
Jul 29 17:53:22.728714 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.728680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5qcr\" (UniqueName: \"kubernetes.io/projected/066e69d4-22ad-4231-b7c9-4b55475efeee-kube-api-access-d5qcr\") pod \"ingress-canary-8mqfn\" (UID: \"066e69d4-22ad-4231-b7c9-4b55475efeee\") " pod="openshift-ingress-canary/ingress-canary-8mqfn"
Jul 29 17:53:22.728714 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.728708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnstr\" (UniqueName: \"kubernetes.io/projected/bedfbcbc-a007-4186-9405-989cdc0afd42-kube-api-access-tnstr\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.728915 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.728761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e5a0c7c-c535-4652-968b-f177289a242e-data-volume\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.728915 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.728786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5e5a0c7c-c535-4652-968b-f177289a242e-crio-socket\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.728915 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.728858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5e5a0c7c-c535-4652-968b-f177289a242e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.728915 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.728892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bplt\" (UniqueName: \"kubernetes.io/projected/5e5a0c7c-c535-4652-968b-f177289a242e-kube-api-access-5bplt\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.728915 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.728914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bedfbcbc-a007-4186-9405-989cdc0afd42-tmp-dir\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.729172 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.728946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bedfbcbc-a007-4186-9405-989cdc0afd42-config-volume\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.732764 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.732738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/066e69d4-22ad-4231-b7c9-4b55475efeee-cert\") pod \"ingress-canary-8mqfn\" (UID: \"066e69d4-22ad-4231-b7c9-4b55475efeee\") " pod="openshift-ingress-canary/ingress-canary-8mqfn"
Jul 29 17:53:22.737463 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.737446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5qcr\" (UniqueName: \"kubernetes.io/projected/066e69d4-22ad-4231-b7c9-4b55475efeee-kube-api-access-d5qcr\") pod \"ingress-canary-8mqfn\" (UID: \"066e69d4-22ad-4231-b7c9-4b55475efeee\") " pod="openshift-ingress-canary/ingress-canary-8mqfn"
Jul 29 17:53:22.829583 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.829395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bedfbcbc-a007-4186-9405-989cdc0afd42-config-volume\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.829756 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.829589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bedfbcbc-a007-4186-9405-989cdc0afd42-metrics-tls\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.829756 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.829608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5e5a0c7c-c535-4652-968b-f177289a242e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.829756 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.829645    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnstr\" (UniqueName: \"kubernetes.io/projected/bedfbcbc-a007-4186-9405-989cdc0afd42-kube-api-access-tnstr\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.829756 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.829681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e5a0c7c-c535-4652-968b-f177289a242e-data-volume\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.829756 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.829706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5e5a0c7c-c535-4652-968b-f177289a242e-crio-socket\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.829756 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.829745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5e5a0c7c-c535-4652-968b-f177289a242e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.830087 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.830011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5e5a0c7c-c535-4652-968b-f177289a242e-crio-socket\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.830087 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.830057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bplt\" (UniqueName: \"kubernetes.io/projected/5e5a0c7c-c535-4652-968b-f177289a242e-kube-api-access-5bplt\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.830087 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.830075    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e5a0c7c-c535-4652-968b-f177289a242e-data-volume\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.830249 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.830092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bedfbcbc-a007-4186-9405-989cdc0afd42-tmp-dir\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.830249 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.830139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bedfbcbc-a007-4186-9405-989cdc0afd42-config-volume\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.830334 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.830276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5e5a0c7c-c535-4652-968b-f177289a242e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.830334 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.830316    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bedfbcbc-a007-4186-9405-989cdc0afd42-tmp-dir\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.832205 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.832183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bedfbcbc-a007-4186-9405-989cdc0afd42-metrics-tls\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.832302 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.832183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5e5a0c7c-c535-4652-968b-f177289a242e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.838603 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.838578    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bplt\" (UniqueName: \"kubernetes.io/projected/5e5a0c7c-c535-4652-968b-f177289a242e-kube-api-access-5bplt\") pod \"insights-runtime-extractor-9nhcg\" (UID: \"5e5a0c7c-c535-4652-968b-f177289a242e\") " pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.838683 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.838626    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnstr\" (UniqueName: \"kubernetes.io/projected/bedfbcbc-a007-4186-9405-989cdc0afd42-kube-api-access-tnstr\") pod \"dns-default-v8gbz\" (UID: \"bedfbcbc-a007-4186-9405-989cdc0afd42\") " pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:22.904608 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.904572    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8mqfn"
Jul 29 17:53:22.927798 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.927767    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9nhcg"
Jul 29 17:53:22.999740 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:22.999708    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:23.123740 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.123710    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9nhcg"]
Jul 29 17:53:23.128553 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.128524    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8mqfn"]
Jul 29 17:53:23.132547 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:23.132524    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod066e69d4_22ad_4231_b7c9_4b55475efeee.slice/crio-3b5077b0cf9c72b96cffe0b8206dc6607aa2ea159707f92a87d21a56442785e2 WatchSource:0}: Error finding container 3b5077b0cf9c72b96cffe0b8206dc6607aa2ea159707f92a87d21a56442785e2: Status 404 returned error can't find the container with id 3b5077b0cf9c72b96cffe0b8206dc6607aa2ea159707f92a87d21a56442785e2
Jul 29 17:53:23.172489 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.172457    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v8gbz"]
Jul 29 17:53:23.176086 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:23.176060    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbedfbcbc_a007_4186_9405_989cdc0afd42.slice/crio-a76fd6b5155635bc1c29c45d1f5c338bff8038f6f585e6766be3108305525857 WatchSource:0}: Error finding container a76fd6b5155635bc1c29c45d1f5c338bff8038f6f585e6766be3108305525857: Status 404 returned error can't find the container with id a76fd6b5155635bc1c29c45d1f5c338bff8038f6f585e6766be3108305525857
Jul 29 17:53:23.267033 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.267005    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:23.267163 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.267005    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:23.267206 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.267013    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:23.274033 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.272166    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w7s6p\""
Jul 29 17:53:23.274033 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.272257    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-llfsb\""
Jul 29 17:53:23.274033 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.272691    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 17:53:23.274033 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.272857    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 17:53:23.274033 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.273063    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 17:53:23.274033 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.273295    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 17:53:23.582869 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.582818    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9nhcg" event={"ID":"5e5a0c7c-c535-4652-968b-f177289a242e","Type":"ContainerStarted","Data":"dcd781e46995746ed0e9507ea8c06dc6941a8c55c62d93d351ac341dc5812cf7"}
Jul 29 17:53:23.584682 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.584655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8mqfn" event={"ID":"066e69d4-22ad-4231-b7c9-4b55475efeee","Type":"ContainerStarted","Data":"3b5077b0cf9c72b96cffe0b8206dc6607aa2ea159707f92a87d21a56442785e2"}
Jul 29 17:53:23.590019 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:23.589986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8gbz" event={"ID":"bedfbcbc-a007-4186-9405-989cdc0afd42","Type":"ContainerStarted","Data":"a76fd6b5155635bc1c29c45d1f5c338bff8038f6f585e6766be3108305525857"}
Jul 29 17:53:24.445811 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.445775    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"]
Jul 29 17:53:24.448058 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.448033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:24.451202 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.451181    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 17:53:24.451416 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.451399    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 29 17:53:24.453010 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.451778    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 17:53:24.453010 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.452010    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 17:53:24.453010 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.452266    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7sg96\""
Jul 29 17:53:24.453010 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.452464    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 29 17:53:24.459944 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.459921    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"]
Jul 29 17:53:24.545092 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.545052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/495dc79f-7292-4d2f-bfc6-203eb16368d3-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:24.545092 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.545097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/495dc79f-7292-4d2f-bfc6-203eb16368d3-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:24.545313 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.545130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj8hv\" (UniqueName: \"kubernetes.io/projected/495dc79f-7292-4d2f-bfc6-203eb16368d3-kube-api-access-jj8hv\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:24.545313 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.545162    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/495dc79f-7292-4d2f-bfc6-203eb16368d3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:24.646113 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.646084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/495dc79f-7292-4d2f-bfc6-203eb16368d3-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:24.646584 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.646151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj8hv\" (UniqueName: \"kubernetes.io/projected/495dc79f-7292-4d2f-bfc6-203eb16368d3-kube-api-access-jj8hv\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:24.646584 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.646186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/495dc79f-7292-4d2f-bfc6-203eb16368d3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:24.646584 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.646282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/495dc79f-7292-4d2f-bfc6-203eb16368d3-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:24.646584 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:24.646394    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 29 17:53:24.646584 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:53:24.646454    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/495dc79f-7292-4d2f-bfc6-203eb16368d3-prometheus-operator-tls podName:495dc79f-7292-4d2f-bfc6-203eb16368d3 nodeName:}" failed. No retries permitted until 2026-07-29 17:53:25.146436011 +0000 UTC m=+56.415934773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/495dc79f-7292-4d2f-bfc6-203eb16368d3-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-m5xkx" (UID: "495dc79f-7292-4d2f-bfc6-203eb16368d3") : secret "prometheus-operator-tls" not found
Jul 29 17:53:24.647009 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.646976    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/495dc79f-7292-4d2f-bfc6-203eb16368d3-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:24.650379 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.650353    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/495dc79f-7292-4d2f-bfc6-203eb16368d3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:24.656160 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:24.656135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj8hv\" (UniqueName: \"kubernetes.io/projected/495dc79f-7292-4d2f-bfc6-203eb16368d3-kube-api-access-jj8hv\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:25.151812 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:25.151629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/495dc79f-7292-4d2f-bfc6-203eb16368d3-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:25.158972 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:25.158915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/495dc79f-7292-4d2f-bfc6-203eb16368d3-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-m5xkx\" (UID: \"495dc79f-7292-4d2f-bfc6-203eb16368d3\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:25.362995 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:25.362937    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"
Jul 29 17:53:29.015309 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:29.015266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-m5xkx"]
Jul 29 17:53:29.019896 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:29.019853    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod495dc79f_7292_4d2f_bfc6_203eb16368d3.slice/crio-70747d5a2d5b619ec463e5290dd1a12017c74c4f9e6461fcdb6a17c15dace4d8 WatchSource:0}: Error finding container 70747d5a2d5b619ec463e5290dd1a12017c74c4f9e6461fcdb6a17c15dace4d8: Status 404 returned error can't find the container with id 70747d5a2d5b619ec463e5290dd1a12017c74c4f9e6461fcdb6a17c15dace4d8
Jul 29 17:53:29.531738 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:29.531714    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ph4nh"
Jul 29 17:53:29.607776 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:29.607738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8mqfn" event={"ID":"066e69d4-22ad-4231-b7c9-4b55475efeee","Type":"ContainerStarted","Data":"4571bdc9284787603a3bac92b0d833ff4779a0a7b8eadf8f3fb4d2cbedf5459a"}
Jul 29 17:53:29.608707 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:29.608684    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx" event={"ID":"495dc79f-7292-4d2f-bfc6-203eb16368d3","Type":"ContainerStarted","Data":"70747d5a2d5b619ec463e5290dd1a12017c74c4f9e6461fcdb6a17c15dace4d8"}
Jul 29 17:53:29.610313 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:29.610289    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8gbz" event={"ID":"bedfbcbc-a007-4186-9405-989cdc0afd42","Type":"ContainerStarted","Data":"01d4f102308e60a954bdc8351567918fa0c4ad8a3c4f61c5f5958d0a592f3106"}
Jul 29 17:53:29.610411 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:29.610341    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8gbz" event={"ID":"bedfbcbc-a007-4186-9405-989cdc0afd42","Type":"ContainerStarted","Data":"1d3360df82a0a35b1fbea53206b0688e0af15c9cc656e8bd754e0bf245f0dc5a"}
Jul 29 17:53:29.610411 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:29.610385    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:29.611772 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:29.611753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9nhcg" event={"ID":"5e5a0c7c-c535-4652-968b-f177289a242e","Type":"ContainerStarted","Data":"46ba32ccdb299b56232a884d07e394e89d86098c25e65d158f8e32b1f2fd518c"}
Jul 29 17:53:29.627521 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:29.627478    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8mqfn" podStartSLOduration=1.897814643 podStartE2EDuration="7.627464673s" totalImagesPullingTime="5.72965003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:29.626766037 +0000 UTC m=+60.896264820" watchObservedRunningTime="2026-07-29 17:53:29.627464673 +0000 UTC m=+60.896963487"
Jul 29 17:53:29.648868 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:29.648813    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v8gbz" podStartSLOduration=1.963333923 podStartE2EDuration="7.648792885s" totalImagesPullingTime="5.685458962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:29.647435821 +0000 UTC m=+60.916934604" watchObservedRunningTime="2026-07-29 17:53:29.648792885 +0000 UTC m=+60.918291669"
Jul 29 17:53:31.619648 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:31.619601    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx" event={"ID":"495dc79f-7292-4d2f-bfc6-203eb16368d3","Type":"ContainerStarted","Data":"53f8aedd433cdaccecdb96ccad286ff640418a621c02358129d6e3c0937ec177"}
Jul 29 17:53:33.628977 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:33.628914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9nhcg" event={"ID":"5e5a0c7c-c535-4652-968b-f177289a242e","Type":"ContainerStarted","Data":"ce2ba121ce7e8052d8c5c9237fa6778a53b0bfc46565d663b804f404a15d8148"}
Jul 29 17:53:33.630991 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:33.630946    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx" event={"ID":"495dc79f-7292-4d2f-bfc6-203eb16368d3","Type":"ContainerStarted","Data":"a6c5455844456554227eba0bef11c5c5962cd56b4df7ea149256d563bc5384a1"}
Jul 29 17:53:33.648588 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:33.648529    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-m5xkx" podStartSLOduration=7.96187753 podStartE2EDuration="9.64851366s" totalImagesPullingTime="1.68663613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:33.645675538 +0000 UTC m=+64.915174322" watchObservedRunningTime="2026-07-29 17:53:33.64851366 +0000 UTC m=+64.918012442"
Jul 29 17:53:34.940328 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:34.940289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:34.940749 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:34.940357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:34.942787 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:34.942763    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 17:53:34.943270 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:34.943233    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 17:53:34.953215 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:34.953191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45f70f28-b309-4c10-ad6b-39ebcc5abcfc-original-pull-secret\") pod \"global-pull-secret-syncer-xmd4d\" (UID: \"45f70f28-b309-4c10-ad6b-39ebcc5abcfc\") " pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:34.953325 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:34.953268    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dc17568-9cb6-4605-bc11-14483b2533e6-metrics-certs\") pod \"network-metrics-daemon-rl5wd\" (UID: \"8dc17568-9cb6-4605-bc11-14483b2533e6\") " pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:34.988492 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:34.988467    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w7s6p\""
Jul 29 17:53:34.990971 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:34.990930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xmd4d"
Jul 29 17:53:34.997243 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:34.997220    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rl5wd"
Jul 29 17:53:35.142122 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.142086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lt72\" (UniqueName: \"kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72\") pod \"network-check-target-wvwkg\" (UID: \"83cac0db-3ee2-4150-859f-1a0a1f720f66\") " pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:35.144231 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.144202    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 17:53:35.154258 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.154237    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 17:53:35.165373 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.165353    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lt72\" (UniqueName: \"kubernetes.io/projected/83cac0db-3ee2-4150-859f-1a0a1f720f66-kube-api-access-6lt72\") pod \"network-check-target-wvwkg\" (UID: \"83cac0db-3ee2-4150-859f-1a0a1f720f66\") " pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:35.282548 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.282475    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-llfsb\""
Jul 29 17:53:35.291197 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.291171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:35.781177 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.781146    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"]
Jul 29 17:53:35.789093 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.789066    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:35.791307 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.791278    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 29 17:53:35.791435 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.791289    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 29 17:53:35.791435 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.791321    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-lm9x6\""
Jul 29 17:53:35.796088 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.796066    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"]
Jul 29 17:53:35.799504 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.799482    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"]
Jul 29 17:53:35.804065 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.804044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:35.807203 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.807183    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 29 17:53:35.807406 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.807202    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-j9thz\""
Jul 29 17:53:35.807406 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.807202    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 29 17:53:35.807675 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.807206    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 29 17:53:35.809540 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.809520    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xh745"]
Jul 29 17:53:35.812645 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.812626    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:35.814512 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.814491    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lwjtf\""
Jul 29 17:53:35.814725 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.814708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 17:53:35.814921 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.814904    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 17:53:35.815834 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.815227    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 17:53:35.818188 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.818167    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"]
Jul 29 17:53:35.948851 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.948809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-root\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:35.948851 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.948851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms856\" (UniqueName: \"kubernetes.io/projected/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-kube-api-access-ms856\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.948877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b344506-3162-458d-b604-ae29eef14fef-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.948896    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c3280d6-4482-492a-8b20-879d7659b158-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.948921    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-accelerators-collector-config\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.948987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgjjp\" (UniqueName: \"kubernetes.io/projected/6c3280d6-4482-492a-8b20-879d7659b158-kube-api-access-lgjjp\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-tls\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-wtmp\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949087    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c3280d6-4482-492a-8b20-879d7659b158-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949112    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b344506-3162-458d-b604-ae29eef14fef-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c3280d6-4482-492a-8b20-879d7659b158-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-sys\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c3280d6-4482-492a-8b20-879d7659b158-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b344506-3162-458d-b604-ae29eef14fef-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgbpr\" (UniqueName: \"kubernetes.io/projected/9b344506-3162-458d-b604-ae29eef14fef-kube-api-access-cgbpr\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-metrics-client-ca\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:35.949441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c3280d6-4482-492a-8b20-879d7659b158-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:35.950236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:35.950236 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:35.949395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-textfile\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.050369 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c3280d6-4482-492a-8b20-879d7659b158-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.050556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.050556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-textfile\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.050556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-root\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.050556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms856\" (UniqueName: \"kubernetes.io/projected/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-kube-api-access-ms856\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.050556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b344506-3162-458d-b604-ae29eef14fef-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:36.050556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c3280d6-4482-492a-8b20-879d7659b158-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.050556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-accelerators-collector-config\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.050877 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6c3280d6-4482-492a-8b20-879d7659b158-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.050877 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjjp\" (UniqueName: \"kubernetes.io/projected/6c3280d6-4482-492a-8b20-879d7659b158-kube-api-access-lgjjp\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.050877 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-tls\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.050877 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-wtmp\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.050877 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c3280d6-4482-492a-8b20-879d7659b158-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.051201 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b344506-3162-458d-b604-ae29eef14fef-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:36.051201 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c3280d6-4482-492a-8b20-879d7659b158-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.051201 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-sys\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.051201 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.050986    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c3280d6-4482-492a-8b20-879d7659b158-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.051201 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.051015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b344506-3162-458d-b604-ae29eef14fef-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:36.051201 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.051036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgbpr\" (UniqueName: \"kubernetes.io/projected/9b344506-3162-458d-b604-ae29eef14fef-kube-api-access-cgbpr\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:36.051201 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.051050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-metrics-client-ca\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.051504 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.051480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-root\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.052067 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.051645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-sys\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.052067 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.051695    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-textfile\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.053485 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.052565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6c3280d6-4482-492a-8b20-879d7659b158-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.053485 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.052638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-metrics-client-ca\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.053485 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.052903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-wtmp\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.053485 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.053433    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-accelerators-collector-config\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.058054 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.054685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-tls\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.058054 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.055179    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b344506-3162-458d-b604-ae29eef14fef-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:36.058054 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.055324    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6c3280d6-4482-492a-8b20-879d7659b158-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.058054 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.055797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b344506-3162-458d-b604-ae29eef14fef-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:36.058054 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.056556    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6c3280d6-4482-492a-8b20-879d7659b158-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.058054 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.057560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b344506-3162-458d-b604-ae29eef14fef-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:36.060023 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.059437    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6c3280d6-4482-492a-8b20-879d7659b158-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.060517 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.060490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.060624 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.060577    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms856\" (UniqueName: \"kubernetes.io/projected/17c7fa6d-3d5d-4903-b359-6bdf57d1200e-kube-api-access-ms856\") pod \"node-exporter-xh745\" (UID: \"17c7fa6d-3d5d-4903-b359-6bdf57d1200e\") " pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.060695 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.060677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgjjp\" (UniqueName: \"kubernetes.io/projected/6c3280d6-4482-492a-8b20-879d7659b158-kube-api-access-lgjjp\") pod \"kube-state-metrics-56d9f944b4-9zfml\" (UID: \"6c3280d6-4482-492a-8b20-879d7659b158\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.066203 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.066181    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgbpr\" (UniqueName: \"kubernetes.io/projected/9b344506-3162-458d-b604-ae29eef14fef-kube-api-access-cgbpr\") pod \"openshift-state-metrics-b8f49f5c6-62dx6\" (UID: \"9b344506-3162-458d-b604-ae29eef14fef\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:36.103815 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.103769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"
Jul 29 17:53:36.116092 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.116063    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"
Jul 29 17:53:36.127183 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.127151    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xh745"
Jul 29 17:53:36.251888 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:36.251852    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c7fa6d_3d5d_4903_b359_6bdf57d1200e.slice/crio-bd7140fb816782fab1d5c943437cd9f92eafa686acc84a5352014f511b08fee3 WatchSource:0}: Error finding container bd7140fb816782fab1d5c943437cd9f92eafa686acc84a5352014f511b08fee3: Status 404 returned error can't find the container with id bd7140fb816782fab1d5c943437cd9f92eafa686acc84a5352014f511b08fee3
Jul 29 17:53:36.412465 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.412404    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rl5wd"]
Jul 29 17:53:36.416437 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:36.416395    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc17568_9cb6_4605_bc11_14483b2533e6.slice/crio-df15a04f21cd9acd7a0b04d5dcd27aef0cec95d67c74a85089aa2dbbcb12144e WatchSource:0}: Error finding container df15a04f21cd9acd7a0b04d5dcd27aef0cec95d67c74a85089aa2dbbcb12144e: Status 404 returned error can't find the container with id df15a04f21cd9acd7a0b04d5dcd27aef0cec95d67c74a85089aa2dbbcb12144e
Jul 29 17:53:36.434537 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.434512    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml"]
Jul 29 17:53:36.438496 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:36.438471    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c3280d6_4482_492a_8b20_879d7659b158.slice/crio-355d11e2a9f95237a5bcd5cacba25efe45d3098acbae8d0bfb9ae6caca9d8b81 WatchSource:0}: Error finding container 355d11e2a9f95237a5bcd5cacba25efe45d3098acbae8d0bfb9ae6caca9d8b81: Status 404 returned error can't find the container with id 355d11e2a9f95237a5bcd5cacba25efe45d3098acbae8d0bfb9ae6caca9d8b81
Jul 29 17:53:36.477596 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.477471    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xmd4d"]
Jul 29 17:53:36.480814 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:36.480785    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45f70f28_b309_4c10_ad6b_39ebcc5abcfc.slice/crio-3cca4cc18d9d700a2305efa86c211ecb86cdff5c73819c6ff3a32ba1facfc3f1 WatchSource:0}: Error finding container 3cca4cc18d9d700a2305efa86c211ecb86cdff5c73819c6ff3a32ba1facfc3f1: Status 404 returned error can't find the container with id 3cca4cc18d9d700a2305efa86c211ecb86cdff5c73819c6ff3a32ba1facfc3f1
Jul 29 17:53:36.643822 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.643747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rl5wd" event={"ID":"8dc17568-9cb6-4605-bc11-14483b2533e6","Type":"ContainerStarted","Data":"df15a04f21cd9acd7a0b04d5dcd27aef0cec95d67c74a85089aa2dbbcb12144e"}
Jul 29 17:53:36.645016 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.644987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xh745" event={"ID":"17c7fa6d-3d5d-4903-b359-6bdf57d1200e","Type":"ContainerStarted","Data":"bd7140fb816782fab1d5c943437cd9f92eafa686acc84a5352014f511b08fee3"}
Jul 29 17:53:36.646252 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.646218    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xmd4d" event={"ID":"45f70f28-b309-4c10-ad6b-39ebcc5abcfc","Type":"ContainerStarted","Data":"3cca4cc18d9d700a2305efa86c211ecb86cdff5c73819c6ff3a32ba1facfc3f1"}
Jul 29 17:53:36.647336 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.647304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml" event={"ID":"6c3280d6-4482-492a-8b20-879d7659b158","Type":"ContainerStarted","Data":"355d11e2a9f95237a5bcd5cacba25efe45d3098acbae8d0bfb9ae6caca9d8b81"}
Jul 29 17:53:36.649105 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.649083    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9nhcg" event={"ID":"5e5a0c7c-c535-4652-968b-f177289a242e","Type":"ContainerStarted","Data":"7dfd8cfe283a2b573c060a9f20b7ae123accc1d6ca1e475f602bae309df2f619"}
Jul 29 17:53:36.657673 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.657654    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wvwkg"]
Jul 29 17:53:36.659146 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.659128    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6"]
Jul 29 17:53:36.660926 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:36.660888    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83cac0db_3ee2_4150_859f_1a0a1f720f66.slice/crio-8ba8f10689189c2d0021c2f1e78a1b4dc570f4b9feae5c094f11e5bd3e072242 WatchSource:0}: Error finding container 8ba8f10689189c2d0021c2f1e78a1b4dc570f4b9feae5c094f11e5bd3e072242: Status 404 returned error can't find the container with id 8ba8f10689189c2d0021c2f1e78a1b4dc570f4b9feae5c094f11e5bd3e072242
Jul 29 17:53:36.662365 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:36.662308    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b344506_3162_458d_b604_ae29eef14fef.slice/crio-6dc04cdb15b0eb59bad8769e929b3f4f923d18bea6770a6b7f4c8cf48c862d71 WatchSource:0}: Error finding container 6dc04cdb15b0eb59bad8769e929b3f4f923d18bea6770a6b7f4c8cf48c862d71: Status 404 returned error can't find the container with id 6dc04cdb15b0eb59bad8769e929b3f4f923d18bea6770a6b7f4c8cf48c862d71
Jul 29 17:53:36.676740 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.676696    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9nhcg" podStartSLOduration=1.517365858 podStartE2EDuration="14.676679201s" totalImagesPullingTime="13.159313343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:36.676285126 +0000 UTC m=+67.945783910" watchObservedRunningTime="2026-07-29 17:53:36.676679201 +0000 UTC m=+67.946177985"
Jul 29 17:53:36.962757 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.962722    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:53:36.986754 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.986682    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:36.988839 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.988815    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:53:36.991078 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.991055    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 17:53:36.991312 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.991056    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 17:53:36.991312 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.991225    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 17:53:36.994160 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.991635    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 17:53:36.994160 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.991767    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 17:53:36.994160 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.991981    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 17:53:36.994160 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.992229    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 17:53:36.994160 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.992527    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4fqbq\""
Jul 29 17:53:36.994160 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.992853    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 17:53:36.994160 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:36.993104    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 17:53:37.059656 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.059621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-volume\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.059842 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.059679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.059842 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.059707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.059842 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.059733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.059842 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.059774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.059842 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.059792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66tjj\" (UniqueName: \"kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-kube-api-access-66tjj\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.059842 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.059830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.060178 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.059859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-web-config\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.060178 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.059884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-out\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.060178 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.059909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.060178 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.059936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.060178 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.060043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.060178 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.060088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161285 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161285 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161467 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161467 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-volume\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161467 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161467 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161467 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161467 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161416    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161467 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66tjj\" (UniqueName: \"kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-kube-api-access-66tjj\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161683 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161473    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161683 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161501    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-web-config\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161683 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161526    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-out\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.161683 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.161550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.162309 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.162200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.162634 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.162610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.163632 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.163574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.165860 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.165422    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.166121 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.166065    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.166694 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.166637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-web-config\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.166817 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.166782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.167630 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.167607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.168163 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.168123    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-out\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.169247 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.169226    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.170467 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.170443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-volume\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.170873 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.170849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.171417 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.171016    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66tjj\" (UniqueName: \"kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-kube-api-access-66tjj\") pod \"alertmanager-main-0\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.301088 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.301054    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:53:37.590903 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.590740    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:53:37.664905 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.664807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wvwkg" event={"ID":"83cac0db-3ee2-4150-859f-1a0a1f720f66","Type":"ContainerStarted","Data":"8ba8f10689189c2d0021c2f1e78a1b4dc570f4b9feae5c094f11e5bd3e072242"}
Jul 29 17:53:37.669212 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.669133    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6" event={"ID":"9b344506-3162-458d-b604-ae29eef14fef","Type":"ContainerStarted","Data":"6d30fb51594f5fe3b56fbbab594cdbe3ab40e2933df8501a06def1aa72ee88e5"}
Jul 29 17:53:37.669212 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.669206    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6" event={"ID":"9b344506-3162-458d-b604-ae29eef14fef","Type":"ContainerStarted","Data":"75e9206374ea0eeb249657bd5980f317b04b5e2f2f9acfd3063fe4764a45cbd6"}
Jul 29 17:53:37.669360 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:37.669239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6" event={"ID":"9b344506-3162-458d-b604-ae29eef14fef","Type":"ContainerStarted","Data":"6dc04cdb15b0eb59bad8769e929b3f4f923d18bea6770a6b7f4c8cf48c862d71"}
Jul 29 17:53:38.382085 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:38.382046    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c963ee_6b4e_4916_a77a_7be4c8f4b85c.slice/crio-0c343d3d0631a66e387f8a5e5a91a0d8e5e5753d9d264b25041a12ad36f597e6 WatchSource:0}: Error finding container 0c343d3d0631a66e387f8a5e5a91a0d8e5e5753d9d264b25041a12ad36f597e6: Status 404 returned error can't find the container with id 0c343d3d0631a66e387f8a5e5a91a0d8e5e5753d9d264b25041a12ad36f597e6
Jul 29 17:53:38.676157 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:38.676072    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerStarted","Data":"0c343d3d0631a66e387f8a5e5a91a0d8e5e5753d9d264b25041a12ad36f597e6"}
Jul 29 17:53:38.949037 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:38.947155    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-84c75594f9-44g4v"]
Jul 29 17:53:38.952179 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:38.952157    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:38.956553 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:38.955908    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lzwll\""
Jul 29 17:53:38.956553 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:38.955936    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 29 17:53:38.956553 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:38.955993    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-f1nbg6qkjaic5\""
Jul 29 17:53:38.956553 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:38.956139    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 29 17:53:38.956553 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:38.956223    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 29 17:53:38.956553 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:38.956363    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 29 17:53:38.956553 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:38.956398    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 29 17:53:38.964553 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:38.964467    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84c75594f9-44g4v"]
Jul 29 17:53:39.081903 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.081598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.081903 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.081642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.081903 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.081668    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-tls\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.081903 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.081688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-grpc-tls\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.081903 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.081727    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.081903 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.081746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89nmj\" (UniqueName: \"kubernetes.io/projected/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-kube-api-access-89nmj\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.081903 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.081765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-metrics-client-ca\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.081903 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.081797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.185491 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.182205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89nmj\" (UniqueName: \"kubernetes.io/projected/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-kube-api-access-89nmj\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.185491 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.182256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-metrics-client-ca\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.185491 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.182287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.185491 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.182368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.185491 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.182417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.185491 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.182542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-tls\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.185491 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.184707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-metrics-client-ca\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.186658 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.186153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-grpc-tls\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.186658 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.186230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.187489 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.187439    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.190115 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.189981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.190115 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.190074    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.190281 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.190250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-tls\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.190862 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.190817    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.191468 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.191440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-secret-grpc-tls\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.196363 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.196341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89nmj\" (UniqueName: \"kubernetes.io/projected/eb63a03f-7d69-4af8-81ac-0b80a5eb9adf-kube-api-access-89nmj\") pod \"thanos-querier-84c75594f9-44g4v\" (UID: \"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf\") " pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.336430 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.336385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:39.365600 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.364222    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-tdn25"]
Jul 29 17:53:39.368219 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.367453    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-tdn25"
Jul 29 17:53:39.370018 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.369929    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 17:53:39.370871 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.370316    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-zstnv\""
Jul 29 17:53:39.371025 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.370381    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 17:53:39.387450 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.387384    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-tdn25"]
Jul 29 17:53:39.489431 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.489402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z52jr\" (UniqueName: \"kubernetes.io/projected/0d7c5d66-1e6d-46d2-b831-522a4a58b3eb-kube-api-access-z52jr\") pod \"downloads-7c97dc4555-tdn25\" (UID: \"0d7c5d66-1e6d-46d2-b831-522a4a58b3eb\") " pod="openshift-console/downloads-7c97dc4555-tdn25"
Jul 29 17:53:39.511434 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.511409    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84c75594f9-44g4v"]
Jul 29 17:53:39.589830 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.589795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z52jr\" (UniqueName: \"kubernetes.io/projected/0d7c5d66-1e6d-46d2-b831-522a4a58b3eb-kube-api-access-z52jr\") pod \"downloads-7c97dc4555-tdn25\" (UID: \"0d7c5d66-1e6d-46d2-b831-522a4a58b3eb\") " pod="openshift-console/downloads-7c97dc4555-tdn25"
Jul 29 17:53:39.602936 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.602909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z52jr\" (UniqueName: \"kubernetes.io/projected/0d7c5d66-1e6d-46d2-b831-522a4a58b3eb-kube-api-access-z52jr\") pod \"downloads-7c97dc4555-tdn25\" (UID: \"0d7c5d66-1e6d-46d2-b831-522a4a58b3eb\") " pod="openshift-console/downloads-7c97dc4555-tdn25"
Jul 29 17:53:39.617396 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.617368    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v8gbz"
Jul 29 17:53:39.683296 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.683265    2580 generic.go:356] "Generic (PLEG): container finished" podID="17c7fa6d-3d5d-4903-b359-6bdf57d1200e" containerID="26dd807f2d61015cbf5e304566bcbbd8bbdfba4895602fcf4a7f42dbd173f5f3" exitCode=0
Jul 29 17:53:39.683447 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.683372    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xh745" event={"ID":"17c7fa6d-3d5d-4903-b359-6bdf57d1200e","Type":"ContainerDied","Data":"26dd807f2d61015cbf5e304566bcbbd8bbdfba4895602fcf4a7f42dbd173f5f3"}
Jul 29 17:53:39.687099 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.687074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml" event={"ID":"6c3280d6-4482-492a-8b20-879d7659b158","Type":"ContainerStarted","Data":"20a97e0eeb1f44277eee48c22d9a99f58fbe8936236b3ce10f84db0aa3e1fdb6"}
Jul 29 17:53:39.687222 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.687132    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml" event={"ID":"6c3280d6-4482-492a-8b20-879d7659b158","Type":"ContainerStarted","Data":"4b672f2547257f20ee6e63dd6b5a7f8f92e7fd06fbdaef11ef26d85e66eed601"}
Jul 29 17:53:39.687222 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.687154    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml" event={"ID":"6c3280d6-4482-492a-8b20-879d7659b158","Type":"ContainerStarted","Data":"565f23beba5c86c423eff704a39971ebbacfa4e3dd08a36361e4500887ec75cb"}
Jul 29 17:53:39.690570 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.690538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6" event={"ID":"9b344506-3162-458d-b604-ae29eef14fef","Type":"ContainerStarted","Data":"15fc191bde4bfeaa6aa8ba4599ce875d19d1feb95b05be2186cc2bb74ad1edd1"}
Jul 29 17:53:39.692465 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.692442    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rl5wd" event={"ID":"8dc17568-9cb6-4605-bc11-14483b2533e6","Type":"ContainerStarted","Data":"4af6c1e030fc9709df4f0048ceb7285e9a3f7ca8355a02138689aa47ff6909bc"}
Jul 29 17:53:39.692575 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.692491    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rl5wd" event={"ID":"8dc17568-9cb6-4605-bc11-14483b2533e6","Type":"ContainerStarted","Data":"ef702b733b602cc8bac60be067e4b62fc2af33352b1d7d1f1c79848c5329a796"}
Jul 29 17:53:39.706792 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.706734    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-tdn25"
Jul 29 17:53:39.719969 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.719905    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-9zfml" podStartSLOduration=2.455919067 podStartE2EDuration="4.71988475s" totalImagesPullingTime="2.263965683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:39.718801085 +0000 UTC m=+70.988299869" watchObservedRunningTime="2026-07-29 17:53:39.71988475 +0000 UTC m=+70.989383537"
Jul 29 17:53:39.734259 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.734210    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rl5wd" podStartSLOduration=68.458226498 podStartE2EDuration="1m10.734193015s" totalImagesPullingTime="2.275966517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:39.732935839 +0000 UTC m=+71.002434623" watchObservedRunningTime="2026-07-29 17:53:39.734193015 +0000 UTC m=+71.003691800"
Jul 29 17:53:39.754486 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:39.754446    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-62dx6" podStartSLOduration=2.913712625 podStartE2EDuration="4.754432757s" totalImagesPullingTime="1.840720132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:39.752669625 +0000 UTC m=+71.022168408" watchObservedRunningTime="2026-07-29 17:53:39.754432757 +0000 UTC m=+71.023931539"
Jul 29 17:53:40.194678 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.194643    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-796596f7bf-nmj46"]
Jul 29 17:53:40.198092 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.197250    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.201184 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.201154    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 29 17:53:40.201306 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.201285    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8c1smp16knp1a\""
Jul 29 17:53:40.201425 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.201379    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 29 17:53:40.201527 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.201498    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fg2j4\""
Jul 29 17:53:40.201628 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.201582    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 29 17:53:40.201852 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.201157    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 17:53:40.210029 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.210006    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-796596f7bf-nmj46"]
Jul 29 17:53:40.296908 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.296881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-audit-log\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.297097 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.296925    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-secret-metrics-server-tls\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.297097 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.296979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-metrics-server-audit-profiles\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.297097 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.297054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-client-ca-bundle\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.297222 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.297153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.297222 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.297195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjmhg\" (UniqueName: \"kubernetes.io/projected/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-kube-api-access-sjmhg\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.297313 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.297233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-secret-metrics-server-client-certs\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.398431 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.398402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-secret-metrics-server-client-certs\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.398870 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.398460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-audit-log\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.398870 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.398506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-secret-metrics-server-tls\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.398870 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.398535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-metrics-server-audit-profiles\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.398870 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.398568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-client-ca-bundle\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.398870 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.398637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.398870 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.398667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjmhg\" (UniqueName: \"kubernetes.io/projected/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-kube-api-access-sjmhg\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.398870 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.398850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-audit-log\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.399542 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.399467    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.399973 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.399930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-metrics-server-audit-profiles\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.401381 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.401353    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-client-ca-bundle\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.401482 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.401465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-secret-metrics-server-client-certs\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.401570 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.401543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-secret-metrics-server-tls\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.408061 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.408034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjmhg\" (UniqueName: \"kubernetes.io/projected/dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9-kube-api-access-sjmhg\") pod \"metrics-server-796596f7bf-nmj46\" (UID: \"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9\") " pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.511194 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.511095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:53:40.559000 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.558949    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"]
Jul 29 17:53:40.561455 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.561435    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"
Jul 29 17:53:40.563598 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.563576    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 29 17:53:40.563704 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.563603    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5j2hb\""
Jul 29 17:53:40.568987 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:40.568929    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb63a03f_7d69_4af8_81ac_0b80a5eb9adf.slice/crio-f018cecd3dea1c96fdf20c6cca1e3da91743792184fe69f58efbe38264d331e4 WatchSource:0}: Error finding container f018cecd3dea1c96fdf20c6cca1e3da91743792184fe69f58efbe38264d331e4: Status 404 returned error can't find the container with id f018cecd3dea1c96fdf20c6cca1e3da91743792184fe69f58efbe38264d331e4
Jul 29 17:53:40.569545 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.569520    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"]
Jul 29 17:53:40.698458 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.698358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v" event={"ID":"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf","Type":"ContainerStarted","Data":"f018cecd3dea1c96fdf20c6cca1e3da91743792184fe69f58efbe38264d331e4"}
Jul 29 17:53:40.700295 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.700268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74qdd\" (UniqueName: \"kubernetes.io/projected/7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37-kube-api-access-74qdd\") pod \"monitoring-plugin-68b7fc9574-qd8n6\" (UID: \"7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"
Jul 29 17:53:40.700435 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.700302    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-qd8n6\" (UID: \"7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"
Jul 29 17:53:40.801639 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.801604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74qdd\" (UniqueName: \"kubernetes.io/projected/7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37-kube-api-access-74qdd\") pod \"monitoring-plugin-68b7fc9574-qd8n6\" (UID: \"7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"
Jul 29 17:53:40.801810 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.801652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-qd8n6\" (UID: \"7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"
Jul 29 17:53:40.807100 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.807070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-qd8n6\" (UID: \"7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"
Jul 29 17:53:40.811567 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.811540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74qdd\" (UniqueName: \"kubernetes.io/projected/7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37-kube-api-access-74qdd\") pod \"monitoring-plugin-68b7fc9574-qd8n6\" (UID: \"7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"
Jul 29 17:53:40.879645 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:40.879616    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"
Jul 29 17:53:42.128992 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.124029    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:53:42.128992 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.128866    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.133255 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.132078    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 17:53:42.133255 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.132460    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 17:53:42.133255 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.132557    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 17:53:42.133255 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.132745    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 17:53:42.133255 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.132881    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3end4gononimi\""
Jul 29 17:53:42.133255 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.133075    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 17:53:42.134259 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.134052    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p9gls\""
Jul 29 17:53:42.134259 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.134129    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 17:53:42.134418 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.134387    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 17:53:42.134543 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.134516    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 17:53:42.135182 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.134760    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 17:53:42.135182 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.134811    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 17:53:42.135182 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.134943    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 17:53:42.135182 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.134764    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 17:53:42.135467 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.135245    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 17:53:42.136469 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.136448    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 17:53:42.146286 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.146264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:53:42.216930 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.216892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217149 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.216977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217149 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217011    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217149 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217149 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217149 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217149 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-web-config\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-config\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6tzc\" (UniqueName: \"kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-kube-api-access-k6tzc\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217349    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-config-out\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.217475 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.217380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.317838 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.317797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.317838 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.317849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.318135 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.317880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.318135 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.317904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6tzc\" (UniqueName: \"kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-kube-api-access-k6tzc\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-config-out\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318508    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318884    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-web-config\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.318942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.319007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.319027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.319061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.319125 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.319082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-config\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.320172 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.319887    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.322390 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.321270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.322390 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.321411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.322390 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.321588    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.322390 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.322125    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.322788 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.322763    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.323426 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.323405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.324428 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.323990    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.324428 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.324253    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.328185 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.328163    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.328313 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.328294    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-web-config\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.328580 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.328560    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.329004 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.328939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-config-out\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.329151 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.329132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.329368 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.329349    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6tzc\" (UniqueName: \"kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-kube-api-access-k6tzc\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.329403 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.329372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-config\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.329633 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.329612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.329755 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.329739    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.331440 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.331417    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.445294 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.445008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:42.678653 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.678596    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-tdn25"]
Jul 29 17:53:42.692928 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.692898    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"]
Jul 29 17:53:42.710355 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.710042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xh745" event={"ID":"17c7fa6d-3d5d-4903-b359-6bdf57d1200e","Type":"ContainerStarted","Data":"fdcd1ffbec8496508008a60e54c053995d81fba62485398cc2695d15496bfe3d"}
Jul 29 17:53:42.710355 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.710119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xh745" event={"ID":"17c7fa6d-3d5d-4903-b359-6bdf57d1200e","Type":"ContainerStarted","Data":"9d87952fbd470f360a15e9d2bbf3e8d0c074b3eef76ce478bd4076bcb1ae5255"}
Jul 29 17:53:42.712535 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.712472    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xmd4d" event={"ID":"45f70f28-b309-4c10-ad6b-39ebcc5abcfc","Type":"ContainerStarted","Data":"ccc9feaf96656eee679bc665bfe4d789d0aa0dab0e3db301392d20f26f4e3065"}
Jul 29 17:53:42.715663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.715192    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wvwkg" event={"ID":"83cac0db-3ee2-4150-859f-1a0a1f720f66","Type":"ContainerStarted","Data":"537b06ada525551dca331b7bf65ef575d012c24110e8827034a0e04ee3cabcb4"}
Jul 29 17:53:42.715663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.715455    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:53:42.716909 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.716888    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-796596f7bf-nmj46"]
Jul 29 17:53:42.717998 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.717944    2580 generic.go:356] "Generic (PLEG): container finished" podID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerID="e3fa1258ca2eedaf8783bb49cfdab7bc02f8e96ba5b02606bedd5dee469d0728" exitCode=0
Jul 29 17:53:42.717998 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.717989    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerDied","Data":"e3fa1258ca2eedaf8783bb49cfdab7bc02f8e96ba5b02606bedd5dee469d0728"}
Jul 29 17:53:42.725610 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:42.725589    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddacbd3e9_ab5a_43cd_9f67_1b382c6df7a9.slice/crio-6d8d71c3d203527a8a46009ca6e2dde43cb38c9f9fb636ef08abf9594f9eacee WatchSource:0}: Error finding container 6d8d71c3d203527a8a46009ca6e2dde43cb38c9f9fb636ef08abf9594f9eacee: Status 404 returned error can't find the container with id 6d8d71c3d203527a8a46009ca6e2dde43cb38c9f9fb636ef08abf9594f9eacee
Jul 29 17:53:42.726376 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:42.726340    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e0d1e83_ce11_46fb_9ac5_74f9f7f61f37.slice/crio-667ec3ca7d4cc0b7f8baf3fdc52aa5107cd8c892b293f502bad94ad5e477f54e WatchSource:0}: Error finding container 667ec3ca7d4cc0b7f8baf3fdc52aa5107cd8c892b293f502bad94ad5e477f54e: Status 404 returned error can't find the container with id 667ec3ca7d4cc0b7f8baf3fdc52aa5107cd8c892b293f502bad94ad5e477f54e
Jul 29 17:53:42.726916 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:42.726896    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d7c5d66_1e6d_46d2_b831_522a4a58b3eb.slice/crio-f9113d6346d389105198a448403ad6abfaa66fd05489e309a8090942f018034e WatchSource:0}: Error finding container f9113d6346d389105198a448403ad6abfaa66fd05489e309a8090942f018034e: Status 404 returned error can't find the container with id f9113d6346d389105198a448403ad6abfaa66fd05489e309a8090942f018034e
Jul 29 17:53:42.731821 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.730877    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xh745" podStartSLOduration=5.14325182 podStartE2EDuration="7.730860143s" totalImagesPullingTime="2.447899359s" totalInitContainerRuntime="139.708964ms" isStatefulPod=true podCreationTimestamp="2026-07-29 17:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:42.729134334 +0000 UTC m=+73.998633121" watchObservedRunningTime="2026-07-29 17:53:42.730860143 +0000 UTC m=+74.000358929"
Jul 29 17:53:42.770453 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.769267    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xmd4d" podStartSLOduration=67.820276359 podStartE2EDuration="1m13.769246839s" totalImagesPullingTime="5.94897048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:42.76731245 +0000 UTC m=+74.036811252" watchObservedRunningTime="2026-07-29 17:53:42.769246839 +0000 UTC m=+74.038745624"
Jul 29 17:53:42.785698 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.785436    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wvwkg" podStartSLOduration=68.026074598 podStartE2EDuration="1m13.785420851s" totalImagesPullingTime="5.759346253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:52:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:42.784249563 +0000 UTC m=+74.053748370" watchObservedRunningTime="2026-07-29 17:53:42.785420851 +0000 UTC m=+74.054919633"
Jul 29 17:53:42.929082 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:42.929034    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:53:42.931305 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:53:42.931279    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd51af197_9c70_44b4_bf58_a36ced863e6f.slice/crio-7d1eae280309a41993ab498eb0e4638c76afa25e3e622bea2e86ba7070060a91 WatchSource:0}: Error finding container 7d1eae280309a41993ab498eb0e4638c76afa25e3e622bea2e86ba7070060a91: Status 404 returned error can't find the container with id 7d1eae280309a41993ab498eb0e4638c76afa25e3e622bea2e86ba7070060a91
Jul 29 17:53:43.731864 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:43.731765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-tdn25" event={"ID":"0d7c5d66-1e6d-46d2-b831-522a4a58b3eb","Type":"ContainerStarted","Data":"f9113d6346d389105198a448403ad6abfaa66fd05489e309a8090942f018034e"}
Jul 29 17:53:43.734545 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:43.734427    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6" event={"ID":"7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37","Type":"ContainerStarted","Data":"667ec3ca7d4cc0b7f8baf3fdc52aa5107cd8c892b293f502bad94ad5e477f54e"}
Jul 29 17:53:43.736141 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:43.736034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-796596f7bf-nmj46" event={"ID":"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9","Type":"ContainerStarted","Data":"6d8d71c3d203527a8a46009ca6e2dde43cb38c9f9fb636ef08abf9594f9eacee"}
Jul 29 17:53:43.740561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:43.739129    2580 generic.go:356] "Generic (PLEG): container finished" podID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerID="d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e" exitCode=0
Jul 29 17:53:43.740561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:43.739891    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerDied","Data":"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"}
Jul 29 17:53:43.740561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:43.739984    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerStarted","Data":"7d1eae280309a41993ab498eb0e4638c76afa25e3e622bea2e86ba7070060a91"}
Jul 29 17:53:46.756991 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.756850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerStarted","Data":"439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2"}
Jul 29 17:53:46.756991 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.756934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerStarted","Data":"2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74"}
Jul 29 17:53:46.757566 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.757538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerStarted","Data":"e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb"}
Jul 29 17:53:46.757643 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.757600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerStarted","Data":"308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260"}
Jul 29 17:53:46.757706 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.757645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerStarted","Data":"d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301"}
Jul 29 17:53:46.759003 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.758566    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6" event={"ID":"7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37","Type":"ContainerStarted","Data":"40bd64f86b44ff35f33f06f3aa8182fbcc4666ba925544607c29f344409bb7ef"}
Jul 29 17:53:46.759003 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.758746    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"
Jul 29 17:53:46.764884 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.762853    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-796596f7bf-nmj46" event={"ID":"dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9","Type":"ContainerStarted","Data":"4df94edef8566ef9d2edbc6463681716a4cd885167171cc6a261da628cb7616f"}
Jul 29 17:53:46.766884 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.766711    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6"
Jul 29 17:53:46.767489 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.767348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v" event={"ID":"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf","Type":"ContainerStarted","Data":"bb0ea88002f068c2048794426b82d455077629f27f2de2852fcdcf7dbadf179e"}
Jul 29 17:53:46.767489 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.767423    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v" event={"ID":"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf","Type":"ContainerStarted","Data":"fedd20fbef29ad3372204055423d16b28e0975db3571bc1f627616006ad345ff"}
Jul 29 17:53:46.767489 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.767458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v" event={"ID":"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf","Type":"ContainerStarted","Data":"d45ed8dcf6e2187103283b836ef3d8e06e91d399a39c6fab9c1558a69225b0f1"}
Jul 29 17:53:46.775270 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.775222    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-qd8n6" podStartSLOduration=3.407610665 podStartE2EDuration="6.775204162s" totalImagesPullingTime="3.367593497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:46.774933426 +0000 UTC m=+78.044432213" watchObservedRunningTime="2026-07-29 17:53:46.775204162 +0000 UTC m=+78.044702947"
Jul 29 17:53:46.792371 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:46.792310    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-796596f7bf-nmj46" podStartSLOduration=3.424751038 podStartE2EDuration="6.792291115s" totalImagesPullingTime="3.367540077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:46.792108063 +0000 UTC m=+78.061606846" watchObservedRunningTime="2026-07-29 17:53:46.792291115 +0000 UTC m=+78.061789900"
Jul 29 17:53:49.786709 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.786669    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerStarted","Data":"71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1"}
Jul 29 17:53:49.790886 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.790848    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerStarted","Data":"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"}
Jul 29 17:53:49.791046 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.790931    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerStarted","Data":"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"}
Jul 29 17:53:49.791046 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.790994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerStarted","Data":"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"}
Jul 29 17:53:49.791046 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.791023    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerStarted","Data":"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"}
Jul 29 17:53:49.791046 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.791046    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerStarted","Data":"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"}
Jul 29 17:53:49.791259 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.791076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerStarted","Data":"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"}
Jul 29 17:53:49.795257 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.795229    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v" event={"ID":"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf","Type":"ContainerStarted","Data":"b1895e9fc731f8a13e149630b5cb38a4e418941e117914273b7e4b3e927777e0"}
Jul 29 17:53:49.795449 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.795301    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v" event={"ID":"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf","Type":"ContainerStarted","Data":"e1433d8825e12b18ded78ca539723e3a569a7357af7dc010523bc77f6500859a"}
Jul 29 17:53:49.795449 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.795344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v" event={"ID":"eb63a03f-7d69-4af8-81ac-0b80a5eb9adf","Type":"ContainerStarted","Data":"326b58d8659846055e5798ef58984c52748b1c6bb38576466587085b3585f4df"}
Jul 29 17:53:49.795449 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.795398    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:53:49.814658 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.814589    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.171745942 podStartE2EDuration="13.814569737s" totalImagesPullingTime="9.605097923s" totalInitContainerRuntime="37.725872ms" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:49.812799819 +0000 UTC m=+81.082298604" watchObservedRunningTime="2026-07-29 17:53:49.814569737 +0000 UTC m=+81.084068520"
Jul 29 17:53:49.842413 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.842355    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.788763109 podStartE2EDuration="7.84233193s" totalImagesPullingTime="5.035566616s" totalInitContainerRuntime="18.002205ms" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:49.840277309 +0000 UTC m=+81.109776095" watchObservedRunningTime="2026-07-29 17:53:49.84233193 +0000 UTC m=+81.111830715"
Jul 29 17:53:49.866250 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:49.866203    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v" podStartSLOduration=3.991502171 podStartE2EDuration="11.86618776s" totalImagesPullingTime="7.874685589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:53:49.863729872 +0000 UTC m=+81.133228679" watchObservedRunningTime="2026-07-29 17:53:49.86618776 +0000 UTC m=+81.135686542"
Jul 29 17:53:52.445608 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:52.445570    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:53:55.808788 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:53:55.808760    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-84c75594f9-44g4v"
Jul 29 17:54:00.512166 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:00.512130    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:54:00.512685 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:00.512196    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:54:02.859022 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:02.858985    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-tdn25" event={"ID":"0d7c5d66-1e6d-46d2-b831-522a4a58b3eb","Type":"ContainerStarted","Data":"cc9681c8eb2a403c5d364b7c7d396fe30a6891c834438042391d1fb32a2c3301"}
Jul 29 17:54:02.859609 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:02.859414    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-tdn25"
Jul 29 17:54:02.874196 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:02.874165    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-tdn25"
Jul 29 17:54:02.875777 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:02.875731    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-tdn25" podStartSLOduration=4.314639318 podStartE2EDuration="23.875714108s" totalImagesPullingTime="19.56107479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:53:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:54:02.875001998 +0000 UTC m=+94.144500806" watchObservedRunningTime="2026-07-29 17:54:02.875714108 +0000 UTC m=+94.145212891"
Jul 29 17:54:13.743099 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:13.743070    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wvwkg"
Jul 29 17:54:20.522558 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:20.522523    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:54:20.527555 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:20.527529    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-796596f7bf-nmj46"
Jul 29 17:54:42.445652 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:42.445615    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:42.505132 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:42.505104    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:43.048948 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:43.048923    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:54:56.164503 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:56.164470    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:54:56.164914 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:56.164805    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="alertmanager" containerID="cri-o://d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301" gracePeriod=120
Jul 29 17:54:56.164914 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:56.164822    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy" containerID="cri-o://2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74" gracePeriod=120
Jul 29 17:54:56.164914 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:56.164841    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="prom-label-proxy" containerID="cri-o://71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1" gracePeriod=120
Jul 29 17:54:56.164914 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:56.164861    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy-web" containerID="cri-o://e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb" gracePeriod=120
Jul 29 17:54:56.164914 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:56.164840    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy-metric" containerID="cri-o://439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2" gracePeriod=120
Jul 29 17:54:56.164914 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:56.164876    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="config-reloader" containerID="cri-o://308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260" gracePeriod=120
Jul 29 17:54:57.094142 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.094116    2580 generic.go:356] "Generic (PLEG): container finished" podID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerID="71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1" exitCode=0
Jul 29 17:54:57.094142 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.094137    2580 generic.go:356] "Generic (PLEG): container finished" podID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerID="439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2" exitCode=0
Jul 29 17:54:57.094142 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.094143    2580 generic.go:356] "Generic (PLEG): container finished" podID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerID="2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74" exitCode=0
Jul 29 17:54:57.094142 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.094149    2580 generic.go:356] "Generic (PLEG): container finished" podID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerID="308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260" exitCode=0
Jul 29 17:54:57.094397 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.094154    2580 generic.go:356] "Generic (PLEG): container finished" podID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerID="d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301" exitCode=0
Jul 29 17:54:57.094397 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.094182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerDied","Data":"71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1"}
Jul 29 17:54:57.094397 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.094245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerDied","Data":"439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2"}
Jul 29 17:54:57.094397 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.094270    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerDied","Data":"2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74"}
Jul 29 17:54:57.094397 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.094295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerDied","Data":"308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260"}
Jul 29 17:54:57.094397 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.094319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerDied","Data":"d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301"}
Jul 29 17:54:57.403810 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.403788    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:57.504758 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.504728    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.504758 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.504764    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-out\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.505003 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.504782    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-main-tls\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.505003 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.504802    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.505003 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.504822    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-cluster-tls-config\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.505003 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.504848    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-tls-assets\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.505003 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.504902    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-web\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.505003 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.504927    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-trusted-ca-bundle\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.505003 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.504982    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-main-db\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.505447 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.505008    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66tjj\" (UniqueName: \"kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-kube-api-access-66tjj\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.505447 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.505076    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-metrics-client-ca\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.505447 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.505110    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-volume\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.505447 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.505141    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-web-config\") pod \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\" (UID: \"09c963ee-6b4e-4916-a77a-7be4c8f4b85c\") "
Jul 29 17:54:57.506351 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.506273    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:54:57.506577 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.506534    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:54:57.506648 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.506605    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:54:57.507845 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.507819    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:54:57.508748 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.508719    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-out" (OuterVolumeSpecName: "config-out") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:54:57.508833 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.508815    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:54:57.509234 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.509195    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:54:57.509367 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.509341    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:54:57.509474 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.509413    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:54:57.509541 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.509487    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-kube-api-access-66tjj" (OuterVolumeSpecName: "kube-api-access-66tjj") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "kube-api-access-66tjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:54:57.509598 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.509572    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-volume" (OuterVolumeSpecName: "config-volume") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:54:57.513567 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.513266    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:54:57.520660 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.520636    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-web-config" (OuterVolumeSpecName: "web-config") pod "09c963ee-6b4e-4916-a77a-7be4c8f4b85c" (UID: "09c963ee-6b4e-4916-a77a-7be4c8f4b85c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:54:57.606427 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606393    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-metrics-client-ca\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606427 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606425    2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-volume\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606454    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-web-config\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606464    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606473    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-config-out\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606482    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-main-tls\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606492    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606502    2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-cluster-tls-config\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606509    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-tls-assets\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606518    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606526    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606534    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-alertmanager-main-db\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:57.606561 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:57.606541    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66tjj\" (UniqueName: \"kubernetes.io/projected/09c963ee-6b4e-4916-a77a-7be4c8f4b85c-kube-api-access-66tjj\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:54:58.101007 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.100978    2580 generic.go:356] "Generic (PLEG): container finished" podID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerID="e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb" exitCode=0
Jul 29 17:54:58.101278 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.101037    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerDied","Data":"e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb"}
Jul 29 17:54:58.101278 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.101069    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.101278 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.101104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"09c963ee-6b4e-4916-a77a-7be4c8f4b85c","Type":"ContainerDied","Data":"0c343d3d0631a66e387f8a5e5a91a0d8e5e5753d9d264b25041a12ad36f597e6"}
Jul 29 17:54:58.101278 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.101137    2580 scope.go:117] "RemoveContainer" containerID="71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1"
Jul 29 17:54:58.109845 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.109828    2580 scope.go:117] "RemoveContainer" containerID="439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2"
Jul 29 17:54:58.117101 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.117082    2580 scope.go:117] "RemoveContainer" containerID="2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74"
Jul 29 17:54:58.124357 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.124336    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:54:58.124850 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.124834    2580 scope.go:117] "RemoveContainer" containerID="e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb"
Jul 29 17:54:58.129386 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.129364    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:54:58.132298 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.132283    2580 scope.go:117] "RemoveContainer" containerID="308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260"
Jul 29 17:54:58.138931 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.138916    2580 scope.go:117] "RemoveContainer" containerID="d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301"
Jul 29 17:54:58.145577 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.145562    2580 scope.go:117] "RemoveContainer" containerID="e3fa1258ca2eedaf8783bb49cfdab7bc02f8e96ba5b02606bedd5dee469d0728"
Jul 29 17:54:58.152248 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.152233    2580 scope.go:117] "RemoveContainer" containerID="71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1"
Jul 29 17:54:58.152520 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:54:58.152498    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1\": container with ID starting with 71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1 not found: ID does not exist" containerID="71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1"
Jul 29 17:54:58.152578 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.152531    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1"} err="failed to get container status \"71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1\": rpc error: code = NotFound desc = could not find container \"71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1\": container with ID starting with 71dac7e61e68636adf96404078c62ffa38cbabdf2251e9b000bebbdfbbaf67c1 not found: ID does not exist"
Jul 29 17:54:58.152578 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.152561    2580 scope.go:117] "RemoveContainer" containerID="439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2"
Jul 29 17:54:58.152792 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:54:58.152777    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2\": container with ID starting with 439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2 not found: ID does not exist" containerID="439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2"
Jul 29 17:54:58.152837 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.152796    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2"} err="failed to get container status \"439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2\": rpc error: code = NotFound desc = could not find container \"439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2\": container with ID starting with 439e477da96cb4d9160436725359102302732e72b5bf0ec5aa75471ae71ba5b2 not found: ID does not exist"
Jul 29 17:54:58.152837 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.152808    2580 scope.go:117] "RemoveContainer" containerID="2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74"
Jul 29 17:54:58.153012 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:54:58.152991    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74\": container with ID starting with 2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74 not found: ID does not exist" containerID="2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74"
Jul 29 17:54:58.153071 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.153019    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74"} err="failed to get container status \"2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74\": rpc error: code = NotFound desc = could not find container \"2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74\": container with ID starting with 2a3af88952e523296cebee65c0c47f96e5688b9183ae6528d1404023df64ba74 not found: ID does not exist"
Jul 29 17:54:58.153071 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.153038    2580 scope.go:117] "RemoveContainer" containerID="e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb"
Jul 29 17:54:58.153291 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:54:58.153272    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb\": container with ID starting with e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb not found: ID does not exist" containerID="e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb"
Jul 29 17:54:58.153354 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.153300    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb"} err="failed to get container status \"e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb\": rpc error: code = NotFound desc = could not find container \"e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb\": container with ID starting with e7a9652d770c61128dfec56d5d4ac45e63b5ce74c4be161f575459e9c229dfcb not found: ID does not exist"
Jul 29 17:54:58.153354 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.153322    2580 scope.go:117] "RemoveContainer" containerID="308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260"
Jul 29 17:54:58.153571 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:54:58.153536    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260\": container with ID starting with 308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260 not found: ID does not exist" containerID="308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260"
Jul 29 17:54:58.153610 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.153577    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260"} err="failed to get container status \"308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260\": rpc error: code = NotFound desc = could not find container \"308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260\": container with ID starting with 308e3caeb11bdec62dc593b3986db66bfc3c2f876292c358d3e4a550ca923260 not found: ID does not exist"
Jul 29 17:54:58.153610 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.153588    2580 scope.go:117] "RemoveContainer" containerID="d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301"
Jul 29 17:54:58.153773 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:54:58.153756    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301\": container with ID starting with d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301 not found: ID does not exist" containerID="d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301"
Jul 29 17:54:58.153821 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.153780    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301"} err="failed to get container status \"d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301\": rpc error: code = NotFound desc = could not find container \"d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301\": container with ID starting with d2c1f51f189b5dd06c0c0661dbf951724627b6b763ad83205446af4128fe8301 not found: ID does not exist"
Jul 29 17:54:58.153821 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.153795    2580 scope.go:117] "RemoveContainer" containerID="e3fa1258ca2eedaf8783bb49cfdab7bc02f8e96ba5b02606bedd5dee469d0728"
Jul 29 17:54:58.154019 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:54:58.154004    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3fa1258ca2eedaf8783bb49cfdab7bc02f8e96ba5b02606bedd5dee469d0728\": container with ID starting with e3fa1258ca2eedaf8783bb49cfdab7bc02f8e96ba5b02606bedd5dee469d0728 not found: ID does not exist" containerID="e3fa1258ca2eedaf8783bb49cfdab7bc02f8e96ba5b02606bedd5dee469d0728"
Jul 29 17:54:58.154069 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.154022    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3fa1258ca2eedaf8783bb49cfdab7bc02f8e96ba5b02606bedd5dee469d0728"} err="failed to get container status \"e3fa1258ca2eedaf8783bb49cfdab7bc02f8e96ba5b02606bedd5dee469d0728\": rpc error: code = NotFound desc = could not find container \"e3fa1258ca2eedaf8783bb49cfdab7bc02f8e96ba5b02606bedd5dee469d0728\": container with ID starting with e3fa1258ca2eedaf8783bb49cfdab7bc02f8e96ba5b02606bedd5dee469d0728 not found: ID does not exist"
Jul 29 17:54:58.158169 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158145    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:54:58.158777 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158759    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="alertmanager"
Jul 29 17:54:58.158869 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158782    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="alertmanager"
Jul 29 17:54:58.158869 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158799    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy-metric"
Jul 29 17:54:58.158869 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158814    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy-metric"
Jul 29 17:54:58.158869 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158838    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="init-config-reloader"
Jul 29 17:54:58.158869 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158847    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="init-config-reloader"
Jul 29 17:54:58.158869 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158866    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="config-reloader"
Jul 29 17:54:58.159153 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158874    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="config-reloader"
Jul 29 17:54:58.159153 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158900    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy"
Jul 29 17:54:58.159153 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158908    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy"
Jul 29 17:54:58.159153 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158927    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy-web"
Jul 29 17:54:58.159153 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.158935    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy-web"
Jul 29 17:54:58.160193 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.160170    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="prom-label-proxy"
Jul 29 17:54:58.160193 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.160196    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="prom-label-proxy"
Jul 29 17:54:58.160677 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.160659    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy-web"
Jul 29 17:54:58.160755 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.160680    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy-metric"
Jul 29 17:54:58.160755 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.160692    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="kube-rbac-proxy"
Jul 29 17:54:58.160755 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.160698    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="alertmanager"
Jul 29 17:54:58.160755 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.160707    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="prom-label-proxy"
Jul 29 17:54:58.160755 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.160713    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" containerName="config-reloader"
Jul 29 17:54:58.166054 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.166037    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.168325 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.168296    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 17:54:58.168325 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.168307    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 17:54:58.168471 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.168307    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 17:54:58.168471 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.168391    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-4fqbq\""
Jul 29 17:54:58.168471 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.168309    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 17:54:58.168590 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.168299    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 17:54:58.168590 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.168548    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 17:54:58.168660 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.168645    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 17:54:58.168738 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.168725    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 17:54:58.174327 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.174293    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 17:54:58.174464 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.174424    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:54:58.210574 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-web-config\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.210683 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210580    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.210683 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.210683 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b6410a58-7371-4e28-ab42-6a03a5daaf71-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.210851 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6410a58-7371-4e28-ab42-6a03a5daaf71-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.210851 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210724    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86sdg\" (UniqueName: \"kubernetes.io/projected/b6410a58-7371-4e28-ab42-6a03a5daaf71-kube-api-access-86sdg\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.210851 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210753    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6410a58-7371-4e28-ab42-6a03a5daaf71-config-out\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.210851 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6410a58-7371-4e28-ab42-6a03a5daaf71-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.211083 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6410a58-7371-4e28-ab42-6a03a5daaf71-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.211083 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-config-volume\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.211083 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.211083 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.210990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.211083 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.211027    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312078 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312186 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-web-config\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312186 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312186 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312350 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b6410a58-7371-4e28-ab42-6a03a5daaf71-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312350 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6410a58-7371-4e28-ab42-6a03a5daaf71-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312452 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86sdg\" (UniqueName: \"kubernetes.io/projected/b6410a58-7371-4e28-ab42-6a03a5daaf71-kube-api-access-86sdg\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312452 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6410a58-7371-4e28-ab42-6a03a5daaf71-config-out\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312452 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6410a58-7371-4e28-ab42-6a03a5daaf71-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312606 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6410a58-7371-4e28-ab42-6a03a5daaf71-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312606 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-config-volume\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312606 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312606 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312561    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b6410a58-7371-4e28-ab42-6a03a5daaf71-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.312606 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.312577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.313780 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.313760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6410a58-7371-4e28-ab42-6a03a5daaf71-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.315192 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.315164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6410a58-7371-4e28-ab42-6a03a5daaf71-config-out\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.315395 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.315372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.315478 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.315454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.315546 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.315505    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6410a58-7371-4e28-ab42-6a03a5daaf71-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.315650 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.315630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.315763 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.315711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-web-config\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.315928 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.315905    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6410a58-7371-4e28-ab42-6a03a5daaf71-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.316097 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.316078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.316605 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.316585    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-config-volume\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.317449 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.317429    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b6410a58-7371-4e28-ab42-6a03a5daaf71-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.322130 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.322112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86sdg\" (UniqueName: \"kubernetes.io/projected/b6410a58-7371-4e28-ab42-6a03a5daaf71-kube-api-access-86sdg\") pod \"alertmanager-main-0\" (UID: \"b6410a58-7371-4e28-ab42-6a03a5daaf71\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.476844 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.476793    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 17:54:58.617308 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:58.617253    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 17:54:58.619405 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:54:58.619381    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6410a58_7371_4e28_ab42_6a03a5daaf71.slice/crio-b001ee0de627d1bccf2f8203b95f06ddfa8e43590754d10e888a57ce21882e0d WatchSource:0}: Error finding container b001ee0de627d1bccf2f8203b95f06ddfa8e43590754d10e888a57ce21882e0d: Status 404 returned error can't find the container with id b001ee0de627d1bccf2f8203b95f06ddfa8e43590754d10e888a57ce21882e0d
Jul 29 17:54:59.107403 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:59.107373    2580 generic.go:356] "Generic (PLEG): container finished" podID="b6410a58-7371-4e28-ab42-6a03a5daaf71" containerID="1cc4282645cefb0c542e471d1feee1d3729dc4f4012417706d75be14b3efd80d" exitCode=0
Jul 29 17:54:59.107560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:59.107440    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6410a58-7371-4e28-ab42-6a03a5daaf71","Type":"ContainerDied","Data":"1cc4282645cefb0c542e471d1feee1d3729dc4f4012417706d75be14b3efd80d"}
Jul 29 17:54:59.107560 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:59.107504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6410a58-7371-4e28-ab42-6a03a5daaf71","Type":"ContainerStarted","Data":"b001ee0de627d1bccf2f8203b95f06ddfa8e43590754d10e888a57ce21882e0d"}
Jul 29 17:54:59.272791 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:54:59.272764    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09c963ee-6b4e-4916-a77a-7be4c8f4b85c" path="/var/lib/kubelet/pods/09c963ee-6b4e-4916-a77a-7be4c8f4b85c/volumes"
Jul 29 17:55:00.116988 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.116811    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6410a58-7371-4e28-ab42-6a03a5daaf71","Type":"ContainerStarted","Data":"1fec1bce907535c47e125b6266b01c3b5f92f955237dc657b61e34a53676551e"}
Jul 29 17:55:00.116988 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.116932    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6410a58-7371-4e28-ab42-6a03a5daaf71","Type":"ContainerStarted","Data":"234c8c391d91e29dbd546cb5f039051a64d40b53619e7eb2a05823f85be5dcd4"}
Jul 29 17:55:00.117440 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.117083    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6410a58-7371-4e28-ab42-6a03a5daaf71","Type":"ContainerStarted","Data":"0ec7e974fc468a63f7c6b5906ba647164c817ae063037cf0bcd24d24976507fd"}
Jul 29 17:55:00.117440 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.117134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6410a58-7371-4e28-ab42-6a03a5daaf71","Type":"ContainerStarted","Data":"4b0d945938b13574ed8f52533bff36d31e2f94073d3b74582773a3f78edb6919"}
Jul 29 17:55:00.117440 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.117171    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6410a58-7371-4e28-ab42-6a03a5daaf71","Type":"ContainerStarted","Data":"1d8e855ec162d13c853f2be36338f222f533c730d4493e82d185a271207ef179"}
Jul 29 17:55:00.117440 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.117207    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6410a58-7371-4e28-ab42-6a03a5daaf71","Type":"ContainerStarted","Data":"0b751f896b217e8ad6c3d497f0e844c228f226e169e775102f28af2d7a5b9299"}
Jul 29 17:55:00.153451 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.153384    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.13573528 podStartE2EDuration="2.15336404s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.62876ms" isStatefulPod=false podCreationTimestamp="2026-07-29 17:54:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:55:00.15017087 +0000 UTC m=+151.419669655" watchObservedRunningTime="2026-07-29 17:55:00.15336404 +0000 UTC m=+151.422862825"
Jul 29 17:55:00.189903 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.189867    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-85445b56b5-j9ww2"]
Jul 29 17:55:00.192727 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.192711    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.197396 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.195375    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 29 17:55:00.197396 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.195763    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 29 17:55:00.197396 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.195947    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jdq52\""
Jul 29 17:55:00.197396 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.196539    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 29 17:55:00.197682 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.197472    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 29 17:55:00.197682 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.197522    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 29 17:55:00.202677 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.202660    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 29 17:55:00.207843 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.207817    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85445b56b5-j9ww2"]
Jul 29 17:55:00.330556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.330523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-telemeter-client-tls\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.330739 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.330564    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef621e30-667c-4e22-b48d-a5aaf92d3a09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.330739 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.330586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef621e30-667c-4e22-b48d-a5aaf92d3a09-serving-certs-ca-bundle\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.330739 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.330652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-secret-telemeter-client\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.330739 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.330711    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-federate-client-tls\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.330739 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.330736    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef621e30-667c-4e22-b48d-a5aaf92d3a09-metrics-client-ca\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.330938 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.330768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.330938 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.330791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2v9z\" (UniqueName: \"kubernetes.io/projected/ef621e30-667c-4e22-b48d-a5aaf92d3a09-kube-api-access-c2v9z\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.431649 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.431563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.431649 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.431602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2v9z\" (UniqueName: \"kubernetes.io/projected/ef621e30-667c-4e22-b48d-a5aaf92d3a09-kube-api-access-c2v9z\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.431841 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.431652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-telemeter-client-tls\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.431841 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.431682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef621e30-667c-4e22-b48d-a5aaf92d3a09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.431841 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.431711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef621e30-667c-4e22-b48d-a5aaf92d3a09-serving-certs-ca-bundle\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.431841 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.431749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-secret-telemeter-client\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.431841 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.431784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-federate-client-tls\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.431841 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.431819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef621e30-667c-4e22-b48d-a5aaf92d3a09-metrics-client-ca\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.432744 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.432707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef621e30-667c-4e22-b48d-a5aaf92d3a09-serving-certs-ca-bundle\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.433084 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.432761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef621e30-667c-4e22-b48d-a5aaf92d3a09-metrics-client-ca\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.433084 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.432925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef621e30-667c-4e22-b48d-a5aaf92d3a09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.434461 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.434438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-telemeter-client-tls\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.434679 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.434663    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-secret-telemeter-client\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.434860 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.434838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.435083 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.435067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ef621e30-667c-4e22-b48d-a5aaf92d3a09-federate-client-tls\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.439357 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.439335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2v9z\" (UniqueName: \"kubernetes.io/projected/ef621e30-667c-4e22-b48d-a5aaf92d3a09-kube-api-access-c2v9z\") pod \"telemeter-client-85445b56b5-j9ww2\" (UID: \"ef621e30-667c-4e22-b48d-a5aaf92d3a09\") " pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.479022 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.478993    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:00.479286 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.479265    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="prometheus" containerID="cri-o://b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833" gracePeriod=600
Jul 29 17:55:00.479367 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.479291    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy" containerID="cri-o://c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9" gracePeriod=600
Jul 29 17:55:00.479367 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.479336    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c" gracePeriod=600
Jul 29 17:55:00.479473 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.479361    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="config-reloader" containerID="cri-o://e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454" gracePeriod=600
Jul 29 17:55:00.479473 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.479366    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="thanos-sidecar" containerID="cri-o://93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1" gracePeriod=600
Jul 29 17:55:00.479473 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.479340    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy-web" containerID="cri-o://e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10" gracePeriod=600
Jul 29 17:55:00.505670 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.505649    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2"
Jul 29 17:55:00.670744 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.670721    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85445b56b5-j9ww2"]
Jul 29 17:55:00.678926 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:55:00.678897    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef621e30_667c_4e22_b48d_a5aaf92d3a09.slice/crio-f7c06c89beb43ffa41bde59cba243632615227cd4e24fd320863a3548ee1e330 WatchSource:0}: Error finding container f7c06c89beb43ffa41bde59cba243632615227cd4e24fd320863a3548ee1e330: Status 404 returned error can't find the container with id f7c06c89beb43ffa41bde59cba243632615227cd4e24fd320863a3548ee1e330
Jul 29 17:55:00.750219 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.750194    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:00.836452 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836421    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6tzc\" (UniqueName: \"kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-kube-api-access-k6tzc\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836620 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836459    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-kubelet-serving-ca-bundle\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836620 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836484    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836620 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836530    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836620 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836570    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-metrics-client-ca\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836620 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836606    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-metrics-client-certs\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836885 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836635    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-2\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836885 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836658    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-grpc-tls\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836885 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-web-config\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836885 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836736    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-tls-assets\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836885 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-serving-certs-ca-bundle\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836885 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836813    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-thanos-prometheus-http-client-file\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836885 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836838    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-config-out\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.836885 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836865    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-1\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.837330 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-db\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.837330 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836926    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-trusted-ca-bundle\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.837330 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.836983    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-config\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.837330 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.837026    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-kube-rbac-proxy\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.837330 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.837039    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:00.837330 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.837060    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-0\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.837330 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.837100    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-tls\") pod \"d51af197-9c70-44b4-bf58-a36ced863e6f\" (UID: \"d51af197-9c70-44b4-bf58-a36ced863e6f\") "
Jul 29 17:55:00.837330 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.837321    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:00.837719 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.837393    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.837719 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.837411    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-metrics-client-ca\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.837823 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.837722    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:00.838377 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.838216    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:00.838523 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.838493    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:00.839255 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.839225    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:00.840232 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.840191    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:55:00.840405 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.840383    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 17:55:00.840697 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.840666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-kube-api-access-k6tzc" (OuterVolumeSpecName: "kube-api-access-k6tzc") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "kube-api-access-k6tzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:55:00.841286 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.841177    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:00.841286 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.841250    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 17:55:00.841431 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.841353    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:00.841506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.841436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-config-out" (OuterVolumeSpecName: "config-out") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 17:55:00.841506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.841450    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:00.841605 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.841592    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:00.841837 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.841818    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:00.842462 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.842436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-config" (OuterVolumeSpecName: "config") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:00.842679 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.842655    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:00.842870 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.842851    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:00.852100 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.852079    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-web-config" (OuterVolumeSpecName: "web-config") pod "d51af197-9c70-44b4-bf58-a36ced863e6f" (UID: "d51af197-9c70-44b4-bf58-a36ced863e6f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 17:55:00.938556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938465    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938506    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-metrics-client-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938518    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-grpc-tls\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938527    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-web-config\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938534    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-tls-assets\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938542    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938551    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938561    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-config-out\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938569    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938925 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938577    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-db\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938925 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938586    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938925 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938594    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-config\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938925 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938602    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-kube-rbac-proxy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938925 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938610    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938925 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938619    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938925 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938627    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6tzc\" (UniqueName: \"kubernetes.io/projected/d51af197-9c70-44b4-bf58-a36ced863e6f-kube-api-access-k6tzc\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938925 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938635    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d51af197-9c70-44b4-bf58-a36ced863e6f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:00.938925 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:00.938644    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d51af197-9c70-44b4-bf58-a36ced863e6f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 17:55:01.123850 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.123817    2580 generic.go:356] "Generic (PLEG): container finished" podID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerID="c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c" exitCode=0
Jul 29 17:55:01.123850 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.123842    2580 generic.go:356] "Generic (PLEG): container finished" podID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerID="c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9" exitCode=0
Jul 29 17:55:01.123850 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.123848    2580 generic.go:356] "Generic (PLEG): container finished" podID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerID="e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10" exitCode=0
Jul 29 17:55:01.123850 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.123855    2580 generic.go:356] "Generic (PLEG): container finished" podID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerID="93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1" exitCode=0
Jul 29 17:55:01.123850 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.123860    2580 generic.go:356] "Generic (PLEG): container finished" podID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerID="e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454" exitCode=0
Jul 29 17:55:01.124506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.123865    2580 generic.go:356] "Generic (PLEG): container finished" podID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerID="b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833" exitCode=0
Jul 29 17:55:01.124506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.123903    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.124506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.123901    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerDied","Data":"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"}
Jul 29 17:55:01.124506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.123991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerDied","Data":"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"}
Jul 29 17:55:01.124506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.124026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerDied","Data":"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"}
Jul 29 17:55:01.124506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.124056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerDied","Data":"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"}
Jul 29 17:55:01.124506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.124079    2580 scope.go:117] "RemoveContainer" containerID="c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"
Jul 29 17:55:01.124506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.124100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerDied","Data":"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"}
Jul 29 17:55:01.124506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.124142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerDied","Data":"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"}
Jul 29 17:55:01.124506 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.124184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d51af197-9c70-44b4-bf58-a36ced863e6f","Type":"ContainerDied","Data":"7d1eae280309a41993ab498eb0e4638c76afa25e3e622bea2e86ba7070060a91"}
Jul 29 17:55:01.125444 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.125418    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2" event={"ID":"ef621e30-667c-4e22-b48d-a5aaf92d3a09","Type":"ContainerStarted","Data":"f7c06c89beb43ffa41bde59cba243632615227cd4e24fd320863a3548ee1e330"}
Jul 29 17:55:01.132501 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.132453    2580 scope.go:117] "RemoveContainer" containerID="c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"
Jul 29 17:55:01.140157 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.140139    2580 scope.go:117] "RemoveContainer" containerID="e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"
Jul 29 17:55:01.147756 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.147688    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:01.147806 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.147784    2580 scope.go:117] "RemoveContainer" containerID="93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"
Jul 29 17:55:01.152439 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.152419    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:01.155264 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.155238    2580 scope.go:117] "RemoveContainer" containerID="e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"
Jul 29 17:55:01.162108 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.162088    2580 scope.go:117] "RemoveContainer" containerID="b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"
Jul 29 17:55:01.169724 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.169693    2580 scope.go:117] "RemoveContainer" containerID="d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"
Jul 29 17:55:01.176718 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.176700    2580 scope.go:117] "RemoveContainer" containerID="c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"
Jul 29 17:55:01.177088 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:55:01.177054    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": container with ID starting with c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c not found: ID does not exist" containerID="c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"
Jul 29 17:55:01.177140 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.177111    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"} err="failed to get container status \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": rpc error: code = NotFound desc = could not find container \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": container with ID starting with c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c not found: ID does not exist"
Jul 29 17:55:01.177140 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.177131    2580 scope.go:117] "RemoveContainer" containerID="c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"
Jul 29 17:55:01.177427 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:55:01.177394    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": container with ID starting with c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9 not found: ID does not exist" containerID="c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"
Jul 29 17:55:01.177578 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.177429    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"} err="failed to get container status \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": rpc error: code = NotFound desc = could not find container \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": container with ID starting with c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9 not found: ID does not exist"
Jul 29 17:55:01.177578 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.177465    2580 scope.go:117] "RemoveContainer" containerID="e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"
Jul 29 17:55:01.177839 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:55:01.177803    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": container with ID starting with e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10 not found: ID does not exist" containerID="e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"
Jul 29 17:55:01.177921 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.177838    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"} err="failed to get container status \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": rpc error: code = NotFound desc = could not find container \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": container with ID starting with e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10 not found: ID does not exist"
Jul 29 17:55:01.177921 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.177859    2580 scope.go:117] "RemoveContainer" containerID="93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"
Jul 29 17:55:01.178242 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:55:01.178201    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": container with ID starting with 93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1 not found: ID does not exist" containerID="93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"
Jul 29 17:55:01.178242 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.178228    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"} err="failed to get container status \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": rpc error: code = NotFound desc = could not find container \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": container with ID starting with 93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1 not found: ID does not exist"
Jul 29 17:55:01.178341 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.178251    2580 scope.go:117] "RemoveContainer" containerID="e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"
Jul 29 17:55:01.178701 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:55:01.178679    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": container with ID starting with e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454 not found: ID does not exist" containerID="e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"
Jul 29 17:55:01.178783 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.178703    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"} err="failed to get container status \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": rpc error: code = NotFound desc = could not find container \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": container with ID starting with e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454 not found: ID does not exist"
Jul 29 17:55:01.178783 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.178716    2580 scope.go:117] "RemoveContainer" containerID="b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"
Jul 29 17:55:01.179002 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:55:01.178983    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": container with ID starting with b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833 not found: ID does not exist" containerID="b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"
Jul 29 17:55:01.179058 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.179010    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"} err="failed to get container status \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": rpc error: code = NotFound desc = could not find container \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": container with ID starting with b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833 not found: ID does not exist"
Jul 29 17:55:01.179058 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.179024    2580 scope.go:117] "RemoveContainer" containerID="d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"
Jul 29 17:55:01.179276 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:55:01.179258    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": container with ID starting with d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e not found: ID does not exist" containerID="d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"
Jul 29 17:55:01.179333 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.179283    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"} err="failed to get container status \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": rpc error: code = NotFound desc = could not find container \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": container with ID starting with d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e not found: ID does not exist"
Jul 29 17:55:01.179333 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.179298    2580 scope.go:117] "RemoveContainer" containerID="c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"
Jul 29 17:55:01.179527 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.179510    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"} err="failed to get container status \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": rpc error: code = NotFound desc = could not find container \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": container with ID starting with c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c not found: ID does not exist"
Jul 29 17:55:01.179571 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.179528    2580 scope.go:117] "RemoveContainer" containerID="c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"
Jul 29 17:55:01.179668 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.179647    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:01.179774 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.179757    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"} err="failed to get container status \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": rpc error: code = NotFound desc = could not find container \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": container with ID starting with c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9 not found: ID does not exist"
Jul 29 17:55:01.179821 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.179777    2580 scope.go:117] "RemoveContainer" containerID="e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"
Jul 29 17:55:01.180030 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180007    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy"
Jul 29 17:55:01.180030 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180027    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180038    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="config-reloader"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180043    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="config-reloader"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180049    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy-web"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180054    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy-web"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180066    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="init-config-reloader"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180071    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="init-config-reloader"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180076    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy-thanos"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180081    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy-thanos"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180003    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"} err="failed to get container status \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": rpc error: code = NotFound desc = could not find container \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": container with ID starting with e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10 not found: ID does not exist"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180095    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="prometheus"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180099    2580 scope.go:117] "RemoveContainer" containerID="93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"
Jul 29 17:55:01.180096 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180103    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="prometheus"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180116    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="thanos-sidecar"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180121    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="thanos-sidecar"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180183    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy-web"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180192    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy-thanos"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180199    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="thanos-sidecar"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180205    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="kube-rbac-proxy"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180212    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="prometheus"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180217    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" containerName="config-reloader"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180301    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"} err="failed to get container status \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": rpc error: code = NotFound desc = could not find container \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": container with ID starting with 93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1 not found: ID does not exist"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180322    2580 scope.go:117] "RemoveContainer" containerID="e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180551    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"} err="failed to get container status \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": rpc error: code = NotFound desc = could not find container \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": container with ID starting with e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454 not found: ID does not exist"
Jul 29 17:55:01.180565 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180564    2580 scope.go:117] "RemoveContainer" containerID="b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"
Jul 29 17:55:01.180939 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180777    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"} err="failed to get container status \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": rpc error: code = NotFound desc = could not find container \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": container with ID starting with b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833 not found: ID does not exist"
Jul 29 17:55:01.180939 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.180795    2580 scope.go:117] "RemoveContainer" containerID="d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"
Jul 29 17:55:01.181083 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.181054    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"} err="failed to get container status \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": rpc error: code = NotFound desc = could not find container \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": container with ID starting with d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e not found: ID does not exist"
Jul 29 17:55:01.181124 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.181085    2580 scope.go:117] "RemoveContainer" containerID="c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"
Jul 29 17:55:01.181327 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.181309    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"} err="failed to get container status \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": rpc error: code = NotFound desc = could not find container \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": container with ID starting with c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c not found: ID does not exist"
Jul 29 17:55:01.181397 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.181328    2580 scope.go:117] "RemoveContainer" containerID="c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"
Jul 29 17:55:01.181571 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.181553    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"} err="failed to get container status \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": rpc error: code = NotFound desc = could not find container \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": container with ID starting with c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9 not found: ID does not exist"
Jul 29 17:55:01.181644 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.181573    2580 scope.go:117] "RemoveContainer" containerID="e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"
Jul 29 17:55:01.181773 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.181757    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"} err="failed to get container status \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": rpc error: code = NotFound desc = could not find container \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": container with ID starting with e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10 not found: ID does not exist"
Jul 29 17:55:01.181840 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.181774    2580 scope.go:117] "RemoveContainer" containerID="93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"
Jul 29 17:55:01.182018 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.181996    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"} err="failed to get container status \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": rpc error: code = NotFound desc = could not find container \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": container with ID starting with 93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1 not found: ID does not exist"
Jul 29 17:55:01.182061 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.182020    2580 scope.go:117] "RemoveContainer" containerID="e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"
Jul 29 17:55:01.182228 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.182209    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"} err="failed to get container status \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": rpc error: code = NotFound desc = could not find container \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": container with ID starting with e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454 not found: ID does not exist"
Jul 29 17:55:01.182274 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.182229    2580 scope.go:117] "RemoveContainer" containerID="b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"
Jul 29 17:55:01.182480 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.182463    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"} err="failed to get container status \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": rpc error: code = NotFound desc = could not find container \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": container with ID starting with b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833 not found: ID does not exist"
Jul 29 17:55:01.182535 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.182480    2580 scope.go:117] "RemoveContainer" containerID="d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"
Jul 29 17:55:01.182688 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.182673    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"} err="failed to get container status \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": rpc error: code = NotFound desc = could not find container \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": container with ID starting with d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e not found: ID does not exist"
Jul 29 17:55:01.182727 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.182688    2580 scope.go:117] "RemoveContainer" containerID="c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"
Jul 29 17:55:01.182893 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.182877    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"} err="failed to get container status \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": rpc error: code = NotFound desc = could not find container \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": container with ID starting with c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c not found: ID does not exist"
Jul 29 17:55:01.182944 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.182893    2580 scope.go:117] "RemoveContainer" containerID="c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"
Jul 29 17:55:01.183116 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.183098    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"} err="failed to get container status \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": rpc error: code = NotFound desc = could not find container \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": container with ID starting with c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9 not found: ID does not exist"
Jul 29 17:55:01.183116 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.183116    2580 scope.go:117] "RemoveContainer" containerID="e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"
Jul 29 17:55:01.183345 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.183306    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"} err="failed to get container status \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": rpc error: code = NotFound desc = could not find container \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": container with ID starting with e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10 not found: ID does not exist"
Jul 29 17:55:01.183393 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.183347    2580 scope.go:117] "RemoveContainer" containerID="93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"
Jul 29 17:55:01.183557 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.183541    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"} err="failed to get container status \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": rpc error: code = NotFound desc = could not find container \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": container with ID starting with 93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1 not found: ID does not exist"
Jul 29 17:55:01.183557 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.183558    2580 scope.go:117] "RemoveContainer" containerID="e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"
Jul 29 17:55:01.183769 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.183744    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"} err="failed to get container status \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": rpc error: code = NotFound desc = could not find container \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": container with ID starting with e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454 not found: ID does not exist"
Jul 29 17:55:01.183769 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.183770    2580 scope.go:117] "RemoveContainer" containerID="b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"
Jul 29 17:55:01.184418 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.184380    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"} err="failed to get container status \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": rpc error: code = NotFound desc = could not find container \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": container with ID starting with b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833 not found: ID does not exist"
Jul 29 17:55:01.184418 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.184418    2580 scope.go:117] "RemoveContainer" containerID="d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"
Jul 29 17:55:01.184651 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.184634    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.185224 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.184904    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"} err="failed to get container status \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": rpc error: code = NotFound desc = could not find container \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": container with ID starting with d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e not found: ID does not exist"
Jul 29 17:55:01.185224 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.184927    2580 scope.go:117] "RemoveContainer" containerID="c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"
Jul 29 17:55:01.185372 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.185270    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"} err="failed to get container status \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": rpc error: code = NotFound desc = could not find container \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": container with ID starting with c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c not found: ID does not exist"
Jul 29 17:55:01.185372 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.185291    2580 scope.go:117] "RemoveContainer" containerID="c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"
Jul 29 17:55:01.185567 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.185549    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"} err="failed to get container status \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": rpc error: code = NotFound desc = could not find container \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": container with ID starting with c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9 not found: ID does not exist"
Jul 29 17:55:01.185612 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.185568    2580 scope.go:117] "RemoveContainer" containerID="e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"
Jul 29 17:55:01.185830 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.185791    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"} err="failed to get container status \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": rpc error: code = NotFound desc = could not find container \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": container with ID starting with e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10 not found: ID does not exist"
Jul 29 17:55:01.185884 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.185831    2580 scope.go:117] "RemoveContainer" containerID="93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"
Jul 29 17:55:01.186135 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.186117    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"} err="failed to get container status \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": rpc error: code = NotFound desc = could not find container \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": container with ID starting with 93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1 not found: ID does not exist"
Jul 29 17:55:01.186207 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.186136    2580 scope.go:117] "RemoveContainer" containerID="e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"
Jul 29 17:55:01.186398 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.186380    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"} err="failed to get container status \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": rpc error: code = NotFound desc = could not find container \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": container with ID starting with e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454 not found: ID does not exist"
Jul 29 17:55:01.186436 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.186400    2580 scope.go:117] "RemoveContainer" containerID="b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"
Jul 29 17:55:01.186642 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.186625    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"} err="failed to get container status \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": rpc error: code = NotFound desc = could not find container \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": container with ID starting with b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833 not found: ID does not exist"
Jul 29 17:55:01.186703 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.186643    2580 scope.go:117] "RemoveContainer" containerID="d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"
Jul 29 17:55:01.186942 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.186918    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"} err="failed to get container status \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": rpc error: code = NotFound desc = could not find container \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": container with ID starting with d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e not found: ID does not exist"
Jul 29 17:55:01.187032 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.186945    2580 scope.go:117] "RemoveContainer" containerID="c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"
Jul 29 17:55:01.187237 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.187215    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c"} err="failed to get container status \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": rpc error: code = NotFound desc = could not find container \"c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c\": container with ID starting with c047ebae1c086d2cceda1dd12a6e2748aff88ca148d95356b605071a27d7698c not found: ID does not exist"
Jul 29 17:55:01.187287 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.187255    2580 scope.go:117] "RemoveContainer" containerID="c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"
Jul 29 17:55:01.187351 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.187336    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 17:55:01.187492 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.187467    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9"} err="failed to get container status \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": rpc error: code = NotFound desc = could not find container \"c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9\": container with ID starting with c61bf80c48e6cdd5a3090795c10f06a8421c54c9262b0381ff06b69811e125a9 not found: ID does not exist"
Jul 29 17:55:01.187556 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.187495    2580 scope.go:117] "RemoveContainer" containerID="e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"
Jul 29 17:55:01.187614 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.187599    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 17:55:01.187656 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.187628    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 17:55:01.187712 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.187697    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 17:55:01.188138 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.188120    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10"} err="failed to get container status \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": rpc error: code = NotFound desc = could not find container \"e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10\": container with ID starting with e9804af35e697694ae57cf1da6020b2ef19aa743db2e72099b4410ba7a752f10 not found: ID does not exist"
Jul 29 17:55:01.188209 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.188139    2580 scope.go:117] "RemoveContainer" containerID="93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"
Jul 29 17:55:01.188346 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.188327    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 17:55:01.188444 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.188402    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1"} err="failed to get container status \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": rpc error: code = NotFound desc = could not find container \"93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1\": container with ID starting with 93fe94e8e32ec63184f5871a71bebf072ff1ed7d8b169afdfc89cc06a61060d1 not found: ID does not exist"
Jul 29 17:55:01.188514 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.188442    2580 scope.go:117] "RemoveContainer" containerID="e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"
Jul 29 17:55:01.188733 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.188687    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454"} err="failed to get container status \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": rpc error: code = NotFound desc = could not find container \"e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454\": container with ID starting with e2f9f1e8d206cc6fec4d5abe0d30693a723bdc6af01629643e94c18709748454 not found: ID does not exist"
Jul 29 17:55:01.188733 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.188709    2580 scope.go:117] "RemoveContainer" containerID="b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"
Jul 29 17:55:01.188997 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.188976    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 17:55:01.188997 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.188988    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833"} err="failed to get container status \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": rpc error: code = NotFound desc = could not find container \"b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833\": container with ID starting with b6f2bfe278a9d28daba6cce3f89c4b044a8e9ba2666d3545eb1c79039120f833 not found: ID does not exist"
Jul 29 17:55:01.189204 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.189010    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 17:55:01.189204 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.189011    2580 scope.go:117] "RemoveContainer" containerID="d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"
Jul 29 17:55:01.189422 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.189400    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3end4gononimi\""
Jul 29 17:55:01.189492 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.189429    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 17:55:01.189667 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.189647    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 17:55:01.189868 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.189846    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p9gls\""
Jul 29 17:55:01.190656 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.190310    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e"} err="failed to get container status \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": rpc error: code = NotFound desc = could not find container \"d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e\": container with ID starting with d9d18def949f7ec10692070b9977b8e849c105f21a82c12789acb35c7d3db13e not found: ID does not exist"
Jul 29 17:55:01.190656 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.190453    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 17:55:01.191642 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.191619    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 17:55:01.193407 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.192564    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 17:55:01.197945 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.197903    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 17:55:01.202222 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.202199    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 17:55:01.203142 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.203118    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:01.241540 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.241507    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.241540 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.241544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-config-out\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.241798 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.241567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.241798 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.241583    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.241908 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.241800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.241992 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.241902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242060 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242060 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242162 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242217 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242217 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242315 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242315 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-web-config\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242315 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242259    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242315 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l898r\" (UniqueName: \"kubernetes.io/projected/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-kube-api-access-l898r\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242315 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242530 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242530 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242467    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242618 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-config\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.242618 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.242582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.271354 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.271321    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d51af197-9c70-44b4-bf58-a36ced863e6f" path="/var/lib/kubelet/pods/d51af197-9c70-44b4-bf58-a36ced863e6f/volumes"
Jul 29 17:55:01.343757 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.343716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.343757 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.343752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344013 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.343778    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344067 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.343999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344067 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344210 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344210 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-web-config\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344210 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344210 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l898r\" (UniqueName: \"kubernetes.io/projected/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-kube-api-access-l898r\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344210 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344473 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344473 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344473 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344325    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-config\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344473 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344473 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344473 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-config-out\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344914 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344489    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344914 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344914 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.344914 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.344612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.347119 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.345292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.347119 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.345384    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.347119 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.345910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.347119 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.346165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.347119 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.346602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.347119 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.346792    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.347509 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.347153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.347509 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.347418    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.347509 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.347432    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.348206 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.348181    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-web-config\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.349416 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.349387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.349642 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.349615    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.349901 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.349881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-config-out\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.349995 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.349920    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.350164 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.350139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.350269 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.350188    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-config\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.350665 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.350645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.351689 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.351672    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.352382 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.352362    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.355932 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.355914    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l898r\" (UniqueName: \"kubernetes.io/projected/cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf-kube-api-access-l898r\") pod \"prometheus-k8s-0\" (UID: \"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.498472 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.498428    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:55:01.661616 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:01.661591    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 17:55:01.664075 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:55:01.664047    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdedfc2b_e5e7_41a0_bdf9_85a8bd42e1bf.slice/crio-3feec14d3d196c364310f5425b0b5384dde15224425dbaadfd6b0b1d7f6d30ea WatchSource:0}: Error finding container 3feec14d3d196c364310f5425b0b5384dde15224425dbaadfd6b0b1d7f6d30ea: Status 404 returned error can't find the container with id 3feec14d3d196c364310f5425b0b5384dde15224425dbaadfd6b0b1d7f6d30ea
Jul 29 17:55:02.133212 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:02.133176    2580 generic.go:356] "Generic (PLEG): container finished" podID="cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf" containerID="bf0f4e47be6659efcf459105ab427d6eb01487d3a399974e1393368ba6e94f72" exitCode=0
Jul 29 17:55:02.133650 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:02.133220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf","Type":"ContainerDied","Data":"bf0f4e47be6659efcf459105ab427d6eb01487d3a399974e1393368ba6e94f72"}
Jul 29 17:55:02.133650 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:02.133275    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf","Type":"ContainerStarted","Data":"3feec14d3d196c364310f5425b0b5384dde15224425dbaadfd6b0b1d7f6d30ea"}
Jul 29 17:55:03.140530 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:03.140429    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2" event={"ID":"ef621e30-667c-4e22-b48d-a5aaf92d3a09","Type":"ContainerStarted","Data":"aa0d12c24e0b9b84956b972970403a99ba9ae9a94228af0e17c38d830ecedd30"}
Jul 29 17:55:03.140932 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:03.140534    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2" event={"ID":"ef621e30-667c-4e22-b48d-a5aaf92d3a09","Type":"ContainerStarted","Data":"83a8f4d57009a7df53f06a6f5d554975ba99d7a13eeceaebebb0739c51168cbe"}
Jul 29 17:55:03.140932 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:03.140560    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2" event={"ID":"ef621e30-667c-4e22-b48d-a5aaf92d3a09","Type":"ContainerStarted","Data":"8eeeeb07ad46c1da639485e83d6f03de5ea466a18202b7cbf741761478b05ab8"}
Jul 29 17:55:03.143510 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:03.143480    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf","Type":"ContainerStarted","Data":"978a33a72e4747a98a4f4cf5e19e8d20d042cedda3d5bba6a1d96546bbc9b8f7"}
Jul 29 17:55:03.143630 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:03.143537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf","Type":"ContainerStarted","Data":"5056ebab6660ba9aff7edba12583c0f35d2c9696f5dde420c40b18b111615b5a"}
Jul 29 17:55:03.143630 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:03.143562    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf","Type":"ContainerStarted","Data":"170a80128ff380f59a36657fe549bdb041b7e400c1604e8220fa44c2f4dcf3f3"}
Jul 29 17:55:03.143630 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:03.143585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf","Type":"ContainerStarted","Data":"2b2429b71328f9ace14a6b177676288dcde45a61345a50c35715c2204f7fedb2"}
Jul 29 17:55:03.143630 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:03.143621    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf","Type":"ContainerStarted","Data":"785b4a5f8c6a449fac01ccfa8184f1759f4183ae2cc74d124fa3b7cc8dc99c71"}
Jul 29 17:55:03.143754 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:03.143643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf","Type":"ContainerStarted","Data":"b1698a0a5fe9434e184c5e3109c1ae45f9fbbb38d2c59327450482dafd6e2f2b"}
Jul 29 17:55:03.162277 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:03.162233    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-85445b56b5-j9ww2" podStartSLOduration=1.228858219 podStartE2EDuration="3.162218732s" totalImagesPullingTime="1.933360513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:55:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:55:03.159613192 +0000 UTC m=+154.429111999" watchObservedRunningTime="2026-07-29 17:55:03.162218732 +0000 UTC m=+154.431717515"
Jul 29 17:55:03.187563 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:03.187512    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.161270095 podStartE2EDuration="2.187495123s" totalImagesPullingTime="0s" totalInitContainerRuntime="26.225028ms" isStatefulPod=false podCreationTimestamp="2026-07-29 17:55:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:55:03.185377843 +0000 UTC m=+154.454876638" watchObservedRunningTime="2026-07-29 17:55:03.187495123 +0000 UTC m=+154.456993951"
Jul 29 17:55:06.499019 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:55:06.498976    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:56:01.499576 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:56:01.499540    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:56:01.515537 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:56:01.515513    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:56:02.392303 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:56:02.392273    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 17:57:29.118045 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:29.118018    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 17:57:54.868568 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:54.868534    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n8kcv"]
Jul 29 17:57:54.870731 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:54.870715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv"
Jul 29 17:57:54.872726 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:54.872707    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 17:57:54.873431 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:54.873413    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 17:57:54.873546 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:54.873420    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-vqmt7\""
Jul 29 17:57:54.878442 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:54.878389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n8kcv"]
Jul 29 17:57:54.953120 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:54.953084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n8kcv\" (UID: \"89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv"
Jul 29 17:57:54.953237 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:54.953154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcsm5\" (UniqueName: \"kubernetes.io/projected/89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0-kube-api-access-pcsm5\") pod \"cert-manager-webhook-6486ffd997-n8kcv\" (UID: \"89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv"
Jul 29 17:57:55.054533 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:55.054492    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcsm5\" (UniqueName: \"kubernetes.io/projected/89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0-kube-api-access-pcsm5\") pod \"cert-manager-webhook-6486ffd997-n8kcv\" (UID: \"89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv"
Jul 29 17:57:55.054663 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:55.054559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n8kcv\" (UID: \"89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv"
Jul 29 17:57:55.062347 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:55.062315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-n8kcv\" (UID: \"89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv"
Jul 29 17:57:55.062441 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:55.062320    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcsm5\" (UniqueName: \"kubernetes.io/projected/89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0-kube-api-access-pcsm5\") pod \"cert-manager-webhook-6486ffd997-n8kcv\" (UID: \"89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv"
Jul 29 17:57:55.195245 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:55.195171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv"
Jul 29 17:57:55.322843 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:55.322794    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-n8kcv"]
Jul 29 17:57:55.325587 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:57:55.325552    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89934fe0_c8fa_4385_bf7d_c7f3dbc21bf0.slice/crio-bef0c8358e844135b258cd2832b0e367fc017e3b324385bfdb0223462610c61a WatchSource:0}: Error finding container bef0c8358e844135b258cd2832b0e367fc017e3b324385bfdb0223462610c61a: Status 404 returned error can't find the container with id bef0c8358e844135b258cd2832b0e367fc017e3b324385bfdb0223462610c61a
Jul 29 17:57:55.327388 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:55.327373    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 17:57:55.825194 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:55.825159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv" event={"ID":"89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0","Type":"ContainerStarted","Data":"bef0c8358e844135b258cd2832b0e367fc017e3b324385bfdb0223462610c61a"}
Jul 29 17:57:57.000768 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:57.000727    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dtr24"]
Jul 29 17:57:57.007633 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:57.007607    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dtr24"
Jul 29 17:57:57.010470 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:57.009848    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-52d6q\""
Jul 29 17:57:57.012996 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:57.012968    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dtr24"]
Jul 29 17:57:57.170899 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:57.170859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/556143f2-f5a9-4674-8f8a-d7e1791fbd98-bound-sa-token\") pod \"cert-manager-55b4d954d5-dtr24\" (UID: \"556143f2-f5a9-4674-8f8a-d7e1791fbd98\") " pod="cert-manager/cert-manager-55b4d954d5-dtr24"
Jul 29 17:57:57.171115 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:57.170943    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2hvn\" (UniqueName: \"kubernetes.io/projected/556143f2-f5a9-4674-8f8a-d7e1791fbd98-kube-api-access-x2hvn\") pod \"cert-manager-55b4d954d5-dtr24\" (UID: \"556143f2-f5a9-4674-8f8a-d7e1791fbd98\") " pod="cert-manager/cert-manager-55b4d954d5-dtr24"
Jul 29 17:57:57.271747 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:57.271670    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2hvn\" (UniqueName: \"kubernetes.io/projected/556143f2-f5a9-4674-8f8a-d7e1791fbd98-kube-api-access-x2hvn\") pod \"cert-manager-55b4d954d5-dtr24\" (UID: \"556143f2-f5a9-4674-8f8a-d7e1791fbd98\") " pod="cert-manager/cert-manager-55b4d954d5-dtr24"
Jul 29 17:57:57.271926 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:57.271783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/556143f2-f5a9-4674-8f8a-d7e1791fbd98-bound-sa-token\") pod \"cert-manager-55b4d954d5-dtr24\" (UID: \"556143f2-f5a9-4674-8f8a-d7e1791fbd98\") " pod="cert-manager/cert-manager-55b4d954d5-dtr24"
Jul 29 17:57:57.281044 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:57.281000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/556143f2-f5a9-4674-8f8a-d7e1791fbd98-bound-sa-token\") pod \"cert-manager-55b4d954d5-dtr24\" (UID: \"556143f2-f5a9-4674-8f8a-d7e1791fbd98\") " pod="cert-manager/cert-manager-55b4d954d5-dtr24"
Jul 29 17:57:57.281342 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:57.281321    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2hvn\" (UniqueName: \"kubernetes.io/projected/556143f2-f5a9-4674-8f8a-d7e1791fbd98-kube-api-access-x2hvn\") pod \"cert-manager-55b4d954d5-dtr24\" (UID: \"556143f2-f5a9-4674-8f8a-d7e1791fbd98\") " pod="cert-manager/cert-manager-55b4d954d5-dtr24"
Jul 29 17:57:57.323067 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:57.323040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-dtr24"
Jul 29 17:57:58.431285 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:58.431251    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-dtr24"]
Jul 29 17:57:58.434108 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:57:58.434081    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod556143f2_f5a9_4674_8f8a_d7e1791fbd98.slice/crio-23d821ed828ca20d3a06314674b055c447bab8f2e9ae72ae77bd07927fe605f1 WatchSource:0}: Error finding container 23d821ed828ca20d3a06314674b055c447bab8f2e9ae72ae77bd07927fe605f1: Status 404 returned error can't find the container with id 23d821ed828ca20d3a06314674b055c447bab8f2e9ae72ae77bd07927fe605f1
Jul 29 17:57:58.844670 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:58.844630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dtr24" event={"ID":"556143f2-f5a9-4674-8f8a-d7e1791fbd98","Type":"ContainerStarted","Data":"4084d85beffbd7ec4de491ea3305fe7487dd5aee1bbe26ab502e0b9e051e7086"}
Jul 29 17:57:58.844863 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:58.844678    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-dtr24" event={"ID":"556143f2-f5a9-4674-8f8a-d7e1791fbd98","Type":"ContainerStarted","Data":"23d821ed828ca20d3a06314674b055c447bab8f2e9ae72ae77bd07927fe605f1"}
Jul 29 17:57:58.846015 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:58.845989    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv" event={"ID":"89934fe0-c8fa-4385-bf7d-c7f3dbc21bf0","Type":"ContainerStarted","Data":"f3f20c0f8c591414aaf946535c50c2854986c5ed08cf6db24e04411949e8dc19"}
Jul 29 17:57:58.846124 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:58.846050    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv"
Jul 29 17:57:58.868393 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:58.868344    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-dtr24" podStartSLOduration=2.868328911 podStartE2EDuration="2.868328911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:57:58.867344872 +0000 UTC m=+330.136843657" watchObservedRunningTime="2026-07-29 17:57:58.868328911 +0000 UTC m=+330.137827672"
Jul 29 17:57:58.888563 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:57:58.888522    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv" podStartSLOduration=1.857668231 podStartE2EDuration="4.888508708s" totalImagesPullingTime="3.030840477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:57:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:57:58.887846703 +0000 UTC m=+330.157345488" watchObservedRunningTime="2026-07-29 17:57:58.888508708 +0000 UTC m=+330.158007491"
Jul 29 17:58:04.852902 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:04.852867    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-n8kcv"
Jul 29 17:58:27.555865 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.555835    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"]
Jul 29 17:58:27.558437 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.558422    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.561150 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.561127    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 17:58:27.561286 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.561201    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 17:58:27.561497 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.561482    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 17:58:27.561717 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.561699    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 17:58:27.561822 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.561809    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9q8v4\""
Jul 29 17:58:27.575694 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.575676    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"]
Jul 29 17:58:27.597494 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.597449    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/620f475b-5a57-46e4-abe0-265834f558ac-tmp\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.597591 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.597497    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/620f475b-5a57-46e4-abe0-265834f558ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.597591 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.597513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd6b2\" (UniqueName: \"kubernetes.io/projected/620f475b-5a57-46e4-abe0-265834f558ac-kube-api-access-hd6b2\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.597591 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.597536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/620f475b-5a57-46e4-abe0-265834f558ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.597591 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.597556    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/620f475b-5a57-46e4-abe0-265834f558ac-manifests\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.697868 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.697850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/620f475b-5a57-46e4-abe0-265834f558ac-tmp\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.698000 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.697897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/620f475b-5a57-46e4-abe0-265834f558ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.698000 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.697914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd6b2\" (UniqueName: \"kubernetes.io/projected/620f475b-5a57-46e4-abe0-265834f558ac-kube-api-access-hd6b2\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.698000 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.697932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/620f475b-5a57-46e4-abe0-265834f558ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.698000 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.697964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/620f475b-5a57-46e4-abe0-265834f558ac-manifests\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.698321 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.698292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/620f475b-5a57-46e4-abe0-265834f558ac-manifests\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.700349 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.700317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/620f475b-5a57-46e4-abe0-265834f558ac-tmp\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.700453 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.700377    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/620f475b-5a57-46e4-abe0-265834f558ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.700513 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.700466    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/620f475b-5a57-46e4-abe0-265834f558ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.706868 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.706839    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd6b2\" (UniqueName: \"kubernetes.io/projected/620f475b-5a57-46e4-abe0-265834f558ac-kube-api-access-hd6b2\") pod \"opendatahub-operator-controller-manager-d84c6c79c-d4fkp\" (UID: \"620f475b-5a57-46e4-abe0-265834f558ac\") " pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:27.869169 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:27.869116    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:28.022111 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:28.021939    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"]
Jul 29 17:58:28.024562 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:58:28.024537    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod620f475b_5a57_46e4_abe0_265834f558ac.slice/crio-0ad5cf354854d7337d23aea3857dcedc31f06357a7809724fccd3d70e2f60da5 WatchSource:0}: Error finding container 0ad5cf354854d7337d23aea3857dcedc31f06357a7809724fccd3d70e2f60da5: Status 404 returned error can't find the container with id 0ad5cf354854d7337d23aea3857dcedc31f06357a7809724fccd3d70e2f60da5
Jul 29 17:58:28.969073 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:28.969028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp" event={"ID":"620f475b-5a57-46e4-abe0-265834f558ac","Type":"ContainerStarted","Data":"0ad5cf354854d7337d23aea3857dcedc31f06357a7809724fccd3d70e2f60da5"}
Jul 29 17:58:31.984487 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:31.984450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp" event={"ID":"620f475b-5a57-46e4-abe0-265834f558ac","Type":"ContainerStarted","Data":"209f92dadeb09907b8ec0412a54322bf9893684e2f2c0b41bc4d62ce29fc163d"}
Jul 29 17:58:32.990764 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:32.990729    2580 generic.go:356] "Generic (PLEG): container finished" podID="620f475b-5a57-46e4-abe0-265834f558ac" containerID="209f92dadeb09907b8ec0412a54322bf9893684e2f2c0b41bc4d62ce29fc163d" exitCode=0
Jul 29 17:58:32.991208 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:32.990822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp" event={"ID":"620f475b-5a57-46e4-abe0-265834f558ac","Type":"ContainerDied","Data":"209f92dadeb09907b8ec0412a54322bf9893684e2f2c0b41bc4d62ce29fc163d"}
Jul 29 17:58:33.997765 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:33.997734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp" event={"ID":"620f475b-5a57-46e4-abe0-265834f558ac","Type":"ContainerStarted","Data":"4acfe07b555b666ae1f1e80e54ecedb07fbb3e7a95ae564af174868ccba0a454"}
Jul 29 17:58:33.998179 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:33.997842    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:34.018709 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:34.018659    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp" podStartSLOduration=2.4133862710000002 podStartE2EDuration="7.01864588s" totalImagesPullingTime="3.556918785s" totalInitContainerRuntime="1.048340824s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:58:34.017263503 +0000 UTC m=+365.286762287" watchObservedRunningTime="2026-07-29 17:58:34.01864588 +0000 UTC m=+365.288144689"
Jul 29 17:58:45.005087 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:45.005057    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d84c6c79c-d4fkp"
Jul 29 17:58:56.196636 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.196603    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"]
Jul 29 17:58:56.199337 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.199318    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.204864 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.204843    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 17:58:56.205214 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.205190    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 17:58:56.205326 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.205200    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 17:58:56.206329 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.206314    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 17:58:56.206573 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.206561    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ddj8l\""
Jul 29 17:58:56.207077 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.207065    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 17:58:56.227299 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.227264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"]
Jul 29 17:58:56.350476 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.350441    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/be34e128-7f01-4b11-93c7-ce5197de6019-metrics-cert\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.350660 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.350490    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/be34e128-7f01-4b11-93c7-ce5197de6019-manager-config\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.350660 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.350568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/be34e128-7f01-4b11-93c7-ce5197de6019-cert\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.350660 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.350596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhc8x\" (UniqueName: \"kubernetes.io/projected/be34e128-7f01-4b11-93c7-ce5197de6019-kube-api-access-zhc8x\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.452056 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.451974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/be34e128-7f01-4b11-93c7-ce5197de6019-cert\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.452056 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.452011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhc8x\" (UniqueName: \"kubernetes.io/projected/be34e128-7f01-4b11-93c7-ce5197de6019-kube-api-access-zhc8x\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.452249 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.452080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/be34e128-7f01-4b11-93c7-ce5197de6019-metrics-cert\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.452249 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.452100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/be34e128-7f01-4b11-93c7-ce5197de6019-manager-config\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.452755 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.452703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/be34e128-7f01-4b11-93c7-ce5197de6019-manager-config\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.454579 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.454557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/be34e128-7f01-4b11-93c7-ce5197de6019-metrics-cert\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.454675 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.454651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/be34e128-7f01-4b11-93c7-ce5197de6019-cert\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.460245 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.460219    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhc8x\" (UniqueName: \"kubernetes.io/projected/be34e128-7f01-4b11-93c7-ce5197de6019-kube-api-access-zhc8x\") pod \"lws-controller-manager-7ff9fc5cb5-c6hw2\" (UID: \"be34e128-7f01-4b11-93c7-ce5197de6019\") " pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.509174 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.509148    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:58:56.660269 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:56.660244    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"]
Jul 29 17:58:56.663263 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:58:56.663231    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe34e128_7f01_4b11_93c7_ce5197de6019.slice/crio-f27dd0bb23da8273f4e8abf2ea18d1542bb22be72996321601927ff259ad3037 WatchSource:0}: Error finding container f27dd0bb23da8273f4e8abf2ea18d1542bb22be72996321601927ff259ad3037: Status 404 returned error can't find the container with id f27dd0bb23da8273f4e8abf2ea18d1542bb22be72996321601927ff259ad3037
Jul 29 17:58:57.098570 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:57.098534    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2" event={"ID":"be34e128-7f01-4b11-93c7-ce5197de6019","Type":"ContainerStarted","Data":"f27dd0bb23da8273f4e8abf2ea18d1542bb22be72996321601927ff259ad3037"}
Jul 29 17:58:58.858515 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.858387    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"]
Jul 29 17:58:58.867075 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.867048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"
Jul 29 17:58:58.869796 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.869646    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-glsc6\""
Jul 29 17:58:58.871346 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.871326    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 29 17:58:58.897154 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.897124    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"]
Jul 29 17:58:58.901981 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.901933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"
Jul 29 17:58:58.907168 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.907147    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 29 17:58:58.907346 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.907327    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ztvq9\""
Jul 29 17:58:58.908995 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.908703    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"]
Jul 29 17:58:58.938296 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.938268    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"]
Jul 29 17:58:58.974753 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.974720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a71e79b4-cd7a-4d57-947e-fdbd388cbd60-cert\") pod \"kserve-controller-manager-d6f4b65b8-jkjr6\" (UID: \"a71e79b4-cd7a-4d57-947e-fdbd388cbd60\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"
Jul 29 17:58:58.974901 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:58.974819    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnjg6\" (UniqueName: \"kubernetes.io/projected/a71e79b4-cd7a-4d57-947e-fdbd388cbd60-kube-api-access-qnjg6\") pod \"kserve-controller-manager-d6f4b65b8-jkjr6\" (UID: \"a71e79b4-cd7a-4d57-947e-fdbd388cbd60\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"
Jul 29 17:58:59.075647 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.075615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnjg6\" (UniqueName: \"kubernetes.io/projected/a71e79b4-cd7a-4d57-947e-fdbd388cbd60-kube-api-access-qnjg6\") pod \"kserve-controller-manager-d6f4b65b8-jkjr6\" (UID: \"a71e79b4-cd7a-4d57-947e-fdbd388cbd60\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"
Jul 29 17:58:59.075841 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.075701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a71e79b4-cd7a-4d57-947e-fdbd388cbd60-cert\") pod \"kserve-controller-manager-d6f4b65b8-jkjr6\" (UID: \"a71e79b4-cd7a-4d57-947e-fdbd388cbd60\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"
Jul 29 17:58:59.075841 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.075726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2897d9aa-2f97-46da-b23f-d7cc6cb28589-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xldbf\" (UID: \"2897d9aa-2f97-46da-b23f-d7cc6cb28589\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"
Jul 29 17:58:59.075841 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.075759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzj2s\" (UniqueName: \"kubernetes.io/projected/2897d9aa-2f97-46da-b23f-d7cc6cb28589-kube-api-access-mzj2s\") pod \"llmisvc-controller-manager-777c6bd7d5-xldbf\" (UID: \"2897d9aa-2f97-46da-b23f-d7cc6cb28589\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"
Jul 29 17:58:59.078634 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.078606    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a71e79b4-cd7a-4d57-947e-fdbd388cbd60-cert\") pod \"kserve-controller-manager-d6f4b65b8-jkjr6\" (UID: \"a71e79b4-cd7a-4d57-947e-fdbd388cbd60\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"
Jul 29 17:58:59.084192 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.084170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnjg6\" (UniqueName: \"kubernetes.io/projected/a71e79b4-cd7a-4d57-947e-fdbd388cbd60-kube-api-access-qnjg6\") pod \"kserve-controller-manager-d6f4b65b8-jkjr6\" (UID: \"a71e79b4-cd7a-4d57-947e-fdbd388cbd60\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"
Jul 29 17:58:59.176535 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.176448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2897d9aa-2f97-46da-b23f-d7cc6cb28589-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xldbf\" (UID: \"2897d9aa-2f97-46da-b23f-d7cc6cb28589\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"
Jul 29 17:58:59.176535 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.176520    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzj2s\" (UniqueName: \"kubernetes.io/projected/2897d9aa-2f97-46da-b23f-d7cc6cb28589-kube-api-access-mzj2s\") pod \"llmisvc-controller-manager-777c6bd7d5-xldbf\" (UID: \"2897d9aa-2f97-46da-b23f-d7cc6cb28589\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"
Jul 29 17:58:59.176736 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:58:59.176618    2580 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 29 17:58:59.176736 ip-10-0-129-228 kubenswrapper[2580]: E0729 17:58:59.176720    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2897d9aa-2f97-46da-b23f-d7cc6cb28589-cert podName:2897d9aa-2f97-46da-b23f-d7cc6cb28589 nodeName:}" failed. No retries permitted until 2026-07-29 17:58:59.676699944 +0000 UTC m=+390.946198711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2897d9aa-2f97-46da-b23f-d7cc6cb28589-cert") pod "llmisvc-controller-manager-777c6bd7d5-xldbf" (UID: "2897d9aa-2f97-46da-b23f-d7cc6cb28589") : secret "llmisvc-webhook-server-cert" not found
Jul 29 17:58:59.180321 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.180284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"
Jul 29 17:58:59.186235 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.186216    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzj2s\" (UniqueName: \"kubernetes.io/projected/2897d9aa-2f97-46da-b23f-d7cc6cb28589-kube-api-access-mzj2s\") pod \"llmisvc-controller-manager-777c6bd7d5-xldbf\" (UID: \"2897d9aa-2f97-46da-b23f-d7cc6cb28589\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"
Jul 29 17:58:59.416066 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.416035    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"]
Jul 29 17:58:59.419512 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:58:59.419487    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda71e79b4_cd7a_4d57_947e_fdbd388cbd60.slice/crio-330b739ad190039b2978e0681f83c764c9f82f303ab72cef0a1f088c4015a352 WatchSource:0}: Error finding container 330b739ad190039b2978e0681f83c764c9f82f303ab72cef0a1f088c4015a352: Status 404 returned error can't find the container with id 330b739ad190039b2978e0681f83c764c9f82f303ab72cef0a1f088c4015a352
Jul 29 17:58:59.681346 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.681304    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2897d9aa-2f97-46da-b23f-d7cc6cb28589-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xldbf\" (UID: \"2897d9aa-2f97-46da-b23f-d7cc6cb28589\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"
Jul 29 17:58:59.683769 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.683744    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2897d9aa-2f97-46da-b23f-d7cc6cb28589-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xldbf\" (UID: \"2897d9aa-2f97-46da-b23f-d7cc6cb28589\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"
Jul 29 17:58:59.812736 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.812707    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"
Jul 29 17:58:59.976273 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:58:59.976225    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2897d9aa_2f97_46da_b23f_d7cc6cb28589.slice/crio-171ff9a1865fa5bc822c8b8adc85fb2663c5c160cbb181e7fe743596629ee53f WatchSource:0}: Error finding container 171ff9a1865fa5bc822c8b8adc85fb2663c5c160cbb181e7fe743596629ee53f: Status 404 returned error can't find the container with id 171ff9a1865fa5bc822c8b8adc85fb2663c5c160cbb181e7fe743596629ee53f
Jul 29 17:58:59.978317 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:58:59.978277    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"]
Jul 29 17:59:00.116880 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:00.116838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2" event={"ID":"be34e128-7f01-4b11-93c7-ce5197de6019","Type":"ContainerStarted","Data":"3f896aa265ef0affb0a2c28d4d5ad5cabcc4ea1cdf0d0826c8b23e0519005478"}
Jul 29 17:59:00.117082 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:00.116996    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:59:00.118455 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:00.118429    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf" event={"ID":"2897d9aa-2f97-46da-b23f-d7cc6cb28589","Type":"ContainerStarted","Data":"171ff9a1865fa5bc822c8b8adc85fb2663c5c160cbb181e7fe743596629ee53f"}
Jul 29 17:59:00.119680 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:00.119659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6" event={"ID":"a71e79b4-cd7a-4d57-947e-fdbd388cbd60","Type":"ContainerStarted","Data":"330b739ad190039b2978e0681f83c764c9f82f303ab72cef0a1f088c4015a352"}
Jul 29 17:59:00.164329 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:00.164281    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2" podStartSLOduration=1.5194974270000001 podStartE2EDuration="4.164266342s" totalImagesPullingTime="2.644768915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:59:00.162456597 +0000 UTC m=+391.431955382" watchObservedRunningTime="2026-07-29 17:59:00.164266342 +0000 UTC m=+391.433765124"
Jul 29 17:59:02.133003 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:02.132894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6" event={"ID":"a71e79b4-cd7a-4d57-947e-fdbd388cbd60","Type":"ContainerStarted","Data":"4a38c125d170e91d60c4d717e29b9f89e4c9dd3e717bf3726b7df427e131823d"}
Jul 29 17:59:02.133505 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:02.133036    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"
Jul 29 17:59:02.163755 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:02.163707    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6" podStartSLOduration=1.7284201559999999 podStartE2EDuration="4.163694268s" totalImagesPullingTime="2.435274112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:59:02.161330208 +0000 UTC m=+393.430828988" watchObservedRunningTime="2026-07-29 17:59:02.163694268 +0000 UTC m=+393.433193052"
Jul 29 17:59:03.139434 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:03.139399    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf" event={"ID":"2897d9aa-2f97-46da-b23f-d7cc6cb28589","Type":"ContainerStarted","Data":"1fa2b4d7ae9e4f0d7b2f59fa881c537d19c2e094115031a1bb603ce519619433"}
Jul 29 17:59:03.139820 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:03.139716    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"
Jul 29 17:59:03.160375 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:03.160327    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf" podStartSLOduration=2.544427086 podStartE2EDuration="5.160313314s" totalImagesPullingTime="2.615886228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:59:03.158581211 +0000 UTC m=+394.428079994" watchObservedRunningTime="2026-07-29 17:59:03.160313314 +0000 UTC m=+394.429812096"
Jul 29 17:59:11.128012 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:11.127977    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7ff9fc5cb5-c6hw2"
Jul 29 17:59:33.145074 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:33.145048    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jkjr6"
Jul 29 17:59:35.149912 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:35.149882    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xldbf"
Jul 29 17:59:49.579414 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:49.579378    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4hv7l"]
Jul 29 17:59:49.586416 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:49.586391    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4hv7l"
Jul 29 17:59:49.588484 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:49.588464    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 17:59:49.588896 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:49.588843    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-4xt2r\""
Jul 29 17:59:49.588896 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:49.588879    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 17:59:49.589122 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:49.589005    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4hv7l"]
Jul 29 17:59:49.701076 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:49.701046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8qpf\" (UniqueName: \"kubernetes.io/projected/d8c03dea-6afd-45fc-9540-aa3ceca004a9-kube-api-access-j8qpf\") pod \"kuadrant-operator-catalog-4hv7l\" (UID: \"d8c03dea-6afd-45fc-9540-aa3ceca004a9\") " pod="kuadrant-system/kuadrant-operator-catalog-4hv7l"
Jul 29 17:59:49.802422 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:49.802390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8qpf\" (UniqueName: \"kubernetes.io/projected/d8c03dea-6afd-45fc-9540-aa3ceca004a9-kube-api-access-j8qpf\") pod \"kuadrant-operator-catalog-4hv7l\" (UID: \"d8c03dea-6afd-45fc-9540-aa3ceca004a9\") " pod="kuadrant-system/kuadrant-operator-catalog-4hv7l"
Jul 29 17:59:49.809381 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:49.809354    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8qpf\" (UniqueName: \"kubernetes.io/projected/d8c03dea-6afd-45fc-9540-aa3ceca004a9-kube-api-access-j8qpf\") pod \"kuadrant-operator-catalog-4hv7l\" (UID: \"d8c03dea-6afd-45fc-9540-aa3ceca004a9\") " pod="kuadrant-system/kuadrant-operator-catalog-4hv7l"
Jul 29 17:59:49.896414 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:49.896326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4hv7l"
Jul 29 17:59:50.026936 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:50.026912    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4hv7l"]
Jul 29 17:59:50.029192 ip-10-0-129-228 kubenswrapper[2580]: W0729 17:59:50.029167    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8c03dea_6afd_45fc_9540_aa3ceca004a9.slice/crio-f2a0409cbce7a75a61858808342877625a6752cfc11c97518919557ecb265b29 WatchSource:0}: Error finding container f2a0409cbce7a75a61858808342877625a6752cfc11c97518919557ecb265b29: Status 404 returned error can't find the container with id f2a0409cbce7a75a61858808342877625a6752cfc11c97518919557ecb265b29
Jul 29 17:59:50.350443 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:50.350402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4hv7l" event={"ID":"d8c03dea-6afd-45fc-9540-aa3ceca004a9","Type":"ContainerStarted","Data":"f2a0409cbce7a75a61858808342877625a6752cfc11c97518919557ecb265b29"}
Jul 29 17:59:52.362815 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:52.362779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4hv7l" event={"ID":"d8c03dea-6afd-45fc-9540-aa3ceca004a9","Type":"ContainerStarted","Data":"9e79bad5a9365ccc420fbe1acc22ef3582ce454002b93bfd833ca6a9ca700d62"}
Jul 29 17:59:52.376991 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:52.376931    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4hv7l" podStartSLOduration=1.6617353769999998 podStartE2EDuration="3.376914525s" totalImagesPullingTime="1.715179148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 17:59:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 17:59:52.37560574 +0000 UTC m=+443.645104525" watchObservedRunningTime="2026-07-29 17:59:52.376914525 +0000 UTC m=+443.646413308"
Jul 29 17:59:59.896478 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:59.896444    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4hv7l"
Jul 29 17:59:59.897079 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:59.896533    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4hv7l"
Jul 29 17:59:59.918402 ip-10-0-129-228 kubenswrapper[2580]: I0729 17:59:59.918379    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4hv7l"
Jul 29 18:00:00.425470 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:00.425444    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4hv7l"
Jul 29 18:00:22.038774 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:22.038735    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"]
Jul 29 18:00:22.045844 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:22.044227    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:22.048101 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:22.048082    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4wqfw\""
Jul 29 18:00:22.064552 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:22.064530    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5xgh\" (UniqueName: \"kubernetes.io/projected/46e6bc6f-cda4-434d-b7e7-adf15eddaac3-kube-api-access-z5xgh\") pod \"limitador-operator-controller-manager-859d7b78bd-6h2wm\" (UID: \"46e6bc6f-cda4-434d-b7e7-adf15eddaac3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:22.068733 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:22.068709    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"]
Jul 29 18:00:22.165346 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:22.165320    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5xgh\" (UniqueName: \"kubernetes.io/projected/46e6bc6f-cda4-434d-b7e7-adf15eddaac3-kube-api-access-z5xgh\") pod \"limitador-operator-controller-manager-859d7b78bd-6h2wm\" (UID: \"46e6bc6f-cda4-434d-b7e7-adf15eddaac3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:22.181197 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:22.181169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5xgh\" (UniqueName: \"kubernetes.io/projected/46e6bc6f-cda4-434d-b7e7-adf15eddaac3-kube-api-access-z5xgh\") pod \"limitador-operator-controller-manager-859d7b78bd-6h2wm\" (UID: \"46e6bc6f-cda4-434d-b7e7-adf15eddaac3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:22.354248 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:22.354185    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:22.485208 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:22.485178    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"]
Jul 29 18:00:22.487555 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:00:22.487528    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e6bc6f_cda4_434d_b7e7_adf15eddaac3.slice/crio-6e8a372f19502d03edb7be87f630ea5d6702bcea9ce7090ccbe8ecb111f79f7d WatchSource:0}: Error finding container 6e8a372f19502d03edb7be87f630ea5d6702bcea9ce7090ccbe8ecb111f79f7d: Status 404 returned error can't find the container with id 6e8a372f19502d03edb7be87f630ea5d6702bcea9ce7090ccbe8ecb111f79f7d
Jul 29 18:00:22.512162 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:22.512138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm" event={"ID":"46e6bc6f-cda4-434d-b7e7-adf15eddaac3","Type":"ContainerStarted","Data":"6e8a372f19502d03edb7be87f630ea5d6702bcea9ce7090ccbe8ecb111f79f7d"}
Jul 29 18:00:24.522970 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:24.522915    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm" event={"ID":"46e6bc6f-cda4-434d-b7e7-adf15eddaac3","Type":"ContainerStarted","Data":"84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c"}
Jul 29 18:00:24.523333 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:24.523070    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:24.544702 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:24.544655    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm" podStartSLOduration=1.6530187760000001 podStartE2EDuration="3.544639061s" totalImagesPullingTime="1.891620285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:00:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:00:24.542352428 +0000 UTC m=+475.811851212" watchObservedRunningTime="2026-07-29 18:00:24.544639061 +0000 UTC m=+475.814137847"
Jul 29 18:00:29.657260 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:29.657225    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kshv2"]
Jul 29 18:00:29.661850 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:29.661829    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kshv2"
Jul 29 18:00:29.663907 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:29.663883    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rjhrp\""
Jul 29 18:00:29.671457 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:29.671436    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kshv2"]
Jul 29 18:00:29.730522 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:29.730496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwwbv\" (UniqueName: \"kubernetes.io/projected/a60a43e0-cd16-436d-bdb3-e22eb7130b9d-kube-api-access-kwwbv\") pod \"authorino-operator-75cc8d5cb7-kshv2\" (UID: \"a60a43e0-cd16-436d-bdb3-e22eb7130b9d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kshv2"
Jul 29 18:00:29.831252 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:29.831220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwwbv\" (UniqueName: \"kubernetes.io/projected/a60a43e0-cd16-436d-bdb3-e22eb7130b9d-kube-api-access-kwwbv\") pod \"authorino-operator-75cc8d5cb7-kshv2\" (UID: \"a60a43e0-cd16-436d-bdb3-e22eb7130b9d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kshv2"
Jul 29 18:00:29.843919 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:29.843888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwwbv\" (UniqueName: \"kubernetes.io/projected/a60a43e0-cd16-436d-bdb3-e22eb7130b9d-kube-api-access-kwwbv\") pod \"authorino-operator-75cc8d5cb7-kshv2\" (UID: \"a60a43e0-cd16-436d-bdb3-e22eb7130b9d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kshv2"
Jul 29 18:00:29.972340 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:29.972249    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kshv2"
Jul 29 18:00:30.138806 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:30.138776    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kshv2"]
Jul 29 18:00:30.142726 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:00:30.142696    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda60a43e0_cd16_436d_bdb3_e22eb7130b9d.slice/crio-d835692c8fa22581752126de85ada1f3818daf0c045308186d43359bd6a70e58 WatchSource:0}: Error finding container d835692c8fa22581752126de85ada1f3818daf0c045308186d43359bd6a70e58: Status 404 returned error can't find the container with id d835692c8fa22581752126de85ada1f3818daf0c045308186d43359bd6a70e58
Jul 29 18:00:30.551885 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:30.551849    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kshv2" event={"ID":"a60a43e0-cd16-436d-bdb3-e22eb7130b9d","Type":"ContainerStarted","Data":"d835692c8fa22581752126de85ada1f3818daf0c045308186d43359bd6a70e58"}
Jul 29 18:00:33.567639 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:33.567603    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kshv2" event={"ID":"a60a43e0-cd16-436d-bdb3-e22eb7130b9d","Type":"ContainerStarted","Data":"5e47155f47aef58deb19c913eec8510931b4817e243ddda9aa6f7ac303d09e36"}
Jul 29 18:00:33.568088 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:33.567685    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kshv2"
Jul 29 18:00:33.586837 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:33.586787    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kshv2" podStartSLOduration=1.822885426 podStartE2EDuration="4.586774313s" totalImagesPullingTime="2.763888887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:00:33.585545593 +0000 UTC m=+484.855044388" watchObservedRunningTime="2026-07-29 18:00:33.586774313 +0000 UTC m=+484.856273096"
Jul 29 18:00:35.528679 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:35.528649    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:44.574247 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:44.574213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kshv2"
Jul 29 18:00:46.664949 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.664915    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"]
Jul 29 18:00:46.665427 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.665188    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm" podUID="46e6bc6f-cda4-434d-b7e7-adf15eddaac3" containerName="manager" containerID="cri-o://84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c" gracePeriod=2
Jul 29 18:00:46.684097 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.684048    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"]
Jul 29 18:00:46.696163 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:00:46.696131    2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"46e6bc6f-cda4-434d-b7e7-adf15eddaac3\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-6h2wm\": pods \"limitador-operator-controller-manager-859d7b78bd-6h2wm\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:46.709335 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.709307    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp"]
Jul 29 18:00:46.709933 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.709914    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46e6bc6f-cda4-434d-b7e7-adf15eddaac3" containerName="manager"
Jul 29 18:00:46.710045 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.709933    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="46e6bc6f-cda4-434d-b7e7-adf15eddaac3" containerName="manager"
Jul 29 18:00:46.710108 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.710066    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="46e6bc6f-cda4-434d-b7e7-adf15eddaac3" containerName="manager"
Jul 29 18:00:46.713342 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.713324    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp"
Jul 29 18:00:46.727821 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.727800    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp"]
Jul 29 18:00:46.743582 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:00:46.743555    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6h2wm\" is forbidden: User \"system:node:ip-10-0-129-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-228.ec2.internal' and this object" podUID="46e6bc6f-cda4-434d-b7e7-adf15eddaac3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:46.777436 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.777409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnzqn\" (UniqueName: \"kubernetes.io/projected/5993b350-bae8-43a3-9c36-48a9a5ff9019-kube-api-access-fnzqn\") pod \"limitador-operator-controller-manager-859d7b78bd-9l6vp\" (UID: \"5993b350-bae8-43a3-9c36-48a9a5ff9019\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp"
Jul 29 18:00:46.878558 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.878528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnzqn\" (UniqueName: \"kubernetes.io/projected/5993b350-bae8-43a3-9c36-48a9a5ff9019-kube-api-access-fnzqn\") pod \"limitador-operator-controller-manager-859d7b78bd-9l6vp\" (UID: \"5993b350-bae8-43a3-9c36-48a9a5ff9019\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp"
Jul 29 18:00:46.887703 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.887681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnzqn\" (UniqueName: \"kubernetes.io/projected/5993b350-bae8-43a3-9c36-48a9a5ff9019-kube-api-access-fnzqn\") pod \"limitador-operator-controller-manager-859d7b78bd-9l6vp\" (UID: \"5993b350-bae8-43a3-9c36-48a9a5ff9019\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp"
Jul 29 18:00:46.910292 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.910270    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:46.911872 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:00:46.911848    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6h2wm\" is forbidden: User \"system:node:ip-10-0-129-228.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-228.ec2.internal' and this object" podUID="46e6bc6f-cda4-434d-b7e7-adf15eddaac3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:46.979403 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.979334    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5xgh\" (UniqueName: \"kubernetes.io/projected/46e6bc6f-cda4-434d-b7e7-adf15eddaac3-kube-api-access-z5xgh\") pod \"46e6bc6f-cda4-434d-b7e7-adf15eddaac3\" (UID: \"46e6bc6f-cda4-434d-b7e7-adf15eddaac3\") "
Jul 29 18:00:46.981606 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:46.981579    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46e6bc6f-cda4-434d-b7e7-adf15eddaac3-kube-api-access-z5xgh" (OuterVolumeSpecName: "kube-api-access-z5xgh") pod "46e6bc6f-cda4-434d-b7e7-adf15eddaac3" (UID: "46e6bc6f-cda4-434d-b7e7-adf15eddaac3"). InnerVolumeSpecName "kube-api-access-z5xgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:00:47.048164 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.048129    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp"
Jul 29 18:00:47.080398 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.080370    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5xgh\" (UniqueName: \"kubernetes.io/projected/46e6bc6f-cda4-434d-b7e7-adf15eddaac3-kube-api-access-z5xgh\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:00:47.196333 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.196305    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp"]
Jul 29 18:00:47.199731 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:00:47.199710    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5993b350_bae8_43a3_9c36_48a9a5ff9019.slice/crio-a2ed4e5a4fab59d64956d0d41953b1654e591dc98a2c8c95b48b26388e77cfeb WatchSource:0}: Error finding container a2ed4e5a4fab59d64956d0d41953b1654e591dc98a2c8c95b48b26388e77cfeb: Status 404 returned error can't find the container with id a2ed4e5a4fab59d64956d0d41953b1654e591dc98a2c8c95b48b26388e77cfeb
Jul 29 18:00:47.273867 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.273844    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46e6bc6f-cda4-434d-b7e7-adf15eddaac3" path="/var/lib/kubelet/pods/46e6bc6f-cda4-434d-b7e7-adf15eddaac3/volumes"
Jul 29 18:00:47.633909 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.633875    2580 generic.go:356] "Generic (PLEG): container finished" podID="46e6bc6f-cda4-434d-b7e7-adf15eddaac3" containerID="84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c" exitCode=0
Jul 29 18:00:47.634107 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.633923    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6h2wm"
Jul 29 18:00:47.634107 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.633998    2580 scope.go:117] "RemoveContainer" containerID="84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c"
Jul 29 18:00:47.635824 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.635795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp" event={"ID":"5993b350-bae8-43a3-9c36-48a9a5ff9019","Type":"ContainerStarted","Data":"e47782de8fd0eae791d2a35da8ba857c0dcbad41a076e88823dc41db465f7c9f"}
Jul 29 18:00:47.635971 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.635850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp" event={"ID":"5993b350-bae8-43a3-9c36-48a9a5ff9019","Type":"ContainerStarted","Data":"a2ed4e5a4fab59d64956d0d41953b1654e591dc98a2c8c95b48b26388e77cfeb"}
Jul 29 18:00:47.636052 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.635993    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp"
Jul 29 18:00:47.642460 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.642445    2580 scope.go:117] "RemoveContainer" containerID="84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c"
Jul 29 18:00:47.642743 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:00:47.642725    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c\": container with ID starting with 84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c not found: ID does not exist" containerID="84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c"
Jul 29 18:00:47.642807 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.642752    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c"} err="failed to get container status \"84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c\": rpc error: code = NotFound desc = could not find container \"84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c\": container with ID starting with 84320e6bf7d5f345eb36a72413ac26f8b6457bc82047203d0a88d5454ae0db5c not found: ID does not exist"
Jul 29 18:00:47.660403 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:47.660362    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp" podStartSLOduration=1.660349209 podStartE2EDuration="1.660349209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:00:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:00:47.657394847 +0000 UTC m=+498.926893632" watchObservedRunningTime="2026-07-29 18:00:47.660349209 +0000 UTC m=+498.929847993"
Jul 29 18:00:58.643857 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:00:58.643826    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9l6vp"
Jul 29 18:01:17.677807 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.677772    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:01:17.683575 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.683554    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"
Jul 29 18:01:17.685425 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.685402    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 18:01:17.685542 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.685461    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dz6bj\""
Jul 29 18:01:17.688407 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.688384    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:01:17.718883 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.718860    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:01:17.734799 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.734779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v2x2\" (UniqueName: \"kubernetes.io/projected/fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80-kube-api-access-4v2x2\") pod \"limitador-limitador-54d6d988d5-w4rbc\" (UID: \"fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"
Jul 29 18:01:17.734915 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.734829    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80-config-file\") pod \"limitador-limitador-54d6d988d5-w4rbc\" (UID: \"fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"
Jul 29 18:01:17.835883 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.835854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v2x2\" (UniqueName: \"kubernetes.io/projected/fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80-kube-api-access-4v2x2\") pod \"limitador-limitador-54d6d988d5-w4rbc\" (UID: \"fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"
Jul 29 18:01:17.836301 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.835914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80-config-file\") pod \"limitador-limitador-54d6d988d5-w4rbc\" (UID: \"fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"
Jul 29 18:01:17.836510 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.836490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80-config-file\") pod \"limitador-limitador-54d6d988d5-w4rbc\" (UID: \"fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"
Jul 29 18:01:17.844636 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.844613    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v2x2\" (UniqueName: \"kubernetes.io/projected/fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80-kube-api-access-4v2x2\") pod \"limitador-limitador-54d6d988d5-w4rbc\" (UID: \"fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"
Jul 29 18:01:17.994740 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:17.994671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"
Jul 29 18:01:18.127925 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:18.127903    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:01:18.130713 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:01:18.130682    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef656ba_b5c3_4bc7_bf9e_5b21d96d0b80.slice/crio-ff6867365e60cee28bcf3ed10e72667af007af63ae28aeb7fe162e0dc876e9a6 WatchSource:0}: Error finding container ff6867365e60cee28bcf3ed10e72667af007af63ae28aeb7fe162e0dc876e9a6: Status 404 returned error can't find the container with id ff6867365e60cee28bcf3ed10e72667af007af63ae28aeb7fe162e0dc876e9a6
Jul 29 18:01:18.795314 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:18.795227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc" event={"ID":"fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80","Type":"ContainerStarted","Data":"ff6867365e60cee28bcf3ed10e72667af007af63ae28aeb7fe162e0dc876e9a6"}
Jul 29 18:01:20.807665 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:20.807632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc" event={"ID":"fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80","Type":"ContainerStarted","Data":"46fe830552e8089df70582f4b465d16dbb2d5b015308949915335dbf969d7216"}
Jul 29 18:01:20.808131 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:20.807716    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"
Jul 29 18:01:20.826899 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:20.826855    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc" podStartSLOduration=1.333054786 podStartE2EDuration="3.826840888s" totalImagesPullingTime="2.493786102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:01:20.825772487 +0000 UTC m=+532.095271289" watchObservedRunningTime="2026-07-29 18:01:20.826840888 +0000 UTC m=+532.096339717"
Jul 29 18:01:31.813513 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:01:31.813481    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"
Jul 29 18:02:33.711025 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:02:33.710996    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:03:04.496224 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:04.496186    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:03:07.392094 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:07.392057    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:03:21.723720 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.723680    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"]
Jul 29 18:03:21.727243 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.727215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.729803 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.729775    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-w4f67\""
Jul 29 18:03:21.729942 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.729777    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 18:03:21.729942 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.729778    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 18:03:21.729942 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.729821    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 29 18:03:21.737623 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.737601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"]
Jul 29 18:03:21.838568 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.838531    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.838770 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.838581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.838770 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.838611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.838770 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.838677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hll6z\" (UniqueName: \"kubernetes.io/projected/1821c52f-9b11-4525-84e5-e1407bf1fec9-kube-api-access-hll6z\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.838914 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.838780    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1821c52f-9b11-4525-84e5-e1407bf1fec9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.838914 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.838808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.940097 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.940055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1821c52f-9b11-4525-84e5-e1407bf1fec9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.940097 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.940100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.940373 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.940123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.940373 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.940177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.940373 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.940213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.940373 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.940233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hll6z\" (UniqueName: \"kubernetes.io/projected/1821c52f-9b11-4525-84e5-e1407bf1fec9-kube-api-access-hll6z\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.940590 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.940489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.940643 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.940598    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.940694 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.940678    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.942633 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.942609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1821c52f-9b11-4525-84e5-e1407bf1fec9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.942898 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.942879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1821c52f-9b11-4525-84e5-e1407bf1fec9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:21.948906 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:21.948881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hll6z\" (UniqueName: \"kubernetes.io/projected/1821c52f-9b11-4525-84e5-e1407bf1fec9-kube-api-access-hll6z\") pod \"e2e-trlp-test-simulated-kserve-598466548d-7dqqd\" (UID: \"1821c52f-9b11-4525-84e5-e1407bf1fec9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:22.037848 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:22.037816    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:22.175916 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:22.175887    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"]
Jul 29 18:03:22.179003 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:03:22.178948    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1821c52f_9b11_4525_84e5_e1407bf1fec9.slice/crio-c27394a2c82615087dce6b11419a88f8313d6b2c3769d0452a9030b63470935d WatchSource:0}: Error finding container c27394a2c82615087dce6b11419a88f8313d6b2c3769d0452a9030b63470935d: Status 404 returned error can't find the container with id c27394a2c82615087dce6b11419a88f8313d6b2c3769d0452a9030b63470935d
Jul 29 18:03:22.180179 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:22.180160    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:03:22.376291 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:22.376207    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd" event={"ID":"1821c52f-9b11-4525-84e5-e1407bf1fec9","Type":"ContainerStarted","Data":"c27394a2c82615087dce6b11419a88f8313d6b2c3769d0452a9030b63470935d"}
Jul 29 18:03:22.994240 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:22.994207    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:03:24.387423 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:24.387384    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd" event={"ID":"1821c52f-9b11-4525-84e5-e1407bf1fec9","Type":"ContainerStarted","Data":"a41b66ceb717a3782e26e3c0984670ccf0705f5f49f882dcf823212799e6187d"}
Jul 29 18:03:24.404194 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:24.404147    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd" podStartSLOduration=2.25929533 podStartE2EDuration="3.404132275s" totalImagesPullingTime="1.144836945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:03:24.402907844 +0000 UTC m=+655.672406638" watchObservedRunningTime="2026-07-29 18:03:24.404132275 +0000 UTC m=+655.673631057"
Jul 29 18:03:32.038004 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:32.037949    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:32.038379 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:32.038036    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:32.050270 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:32.050239    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:03:32.437736 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:03:32.437655    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-7dqqd"
Jul 29 18:04:05.102468 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:05.102431    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:04:10.126655 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.126611    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"]
Jul 29 18:04:10.130359 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.130337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.132337 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.132316    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 29 18:04:10.138337 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.138308    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"]
Jul 29 18:04:10.171193 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.171160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.171358 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.171225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.171358 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.171253    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp7h6\" (UniqueName: \"kubernetes.io/projected/f6875645-0f4d-457a-8947-de8b8d309e09-kube-api-access-kp7h6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.171358 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.171346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6875645-0f4d-457a-8947-de8b8d309e09-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.171483 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.171404    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.171483 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.171433    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.272163 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.272125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.272163 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.272169    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.272407 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.272196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.272407 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.272239    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.272407 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.272266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp7h6\" (UniqueName: \"kubernetes.io/projected/f6875645-0f4d-457a-8947-de8b8d309e09-kube-api-access-kp7h6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.272407 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.272340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6875645-0f4d-457a-8947-de8b8d309e09-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.272681 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.272654    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.272742 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.272663    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.272742 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.272702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.274540 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.274513    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6875645-0f4d-457a-8947-de8b8d309e09-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.274813 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.274796    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6875645-0f4d-457a-8947-de8b8d309e09-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.279728 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.279705    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp7h6\" (UniqueName: \"kubernetes.io/projected/f6875645-0f4d-457a-8947-de8b8d309e09-kube-api-access-kp7h6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-n46nx\" (UID: \"f6875645-0f4d-457a-8947-de8b8d309e09\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.441259 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.441190    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:10.575176 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.575153    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"]
Jul 29 18:04:10.577865 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:04:10.577836    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6875645_0f4d_457a_8947_de8b8d309e09.slice/crio-aca528b26b33cac2df271195ef8f9cb36e5493bdc6d0fa72a2bed8502cd30f82 WatchSource:0}: Error finding container aca528b26b33cac2df271195ef8f9cb36e5493bdc6d0fa72a2bed8502cd30f82: Status 404 returned error can't find the container with id aca528b26b33cac2df271195ef8f9cb36e5493bdc6d0fa72a2bed8502cd30f82
Jul 29 18:04:10.611990 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:10.611941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx" event={"ID":"f6875645-0f4d-457a-8947-de8b8d309e09","Type":"ContainerStarted","Data":"aca528b26b33cac2df271195ef8f9cb36e5493bdc6d0fa72a2bed8502cd30f82"}
Jul 29 18:04:11.618204 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:11.618163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx" event={"ID":"f6875645-0f4d-457a-8947-de8b8d309e09","Type":"ContainerStarted","Data":"5dbaa04bce9f9a65dd9568b1cdcabe187d0b4afcf3da0cfb3b90fde87babe537"}
Jul 29 18:04:11.634533 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:11.634471    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx" podStartSLOduration=1.336256689 podStartE2EDuration="1.63445313s" totalImagesPullingTime="298.196441ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:04:11.633231809 +0000 UTC m=+702.902730592" watchObservedRunningTime="2026-07-29 18:04:11.63445313 +0000 UTC m=+702.903951917"
Jul 29 18:04:12.293744 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:12.293714    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:04:20.441569 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:20.441496    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:20.441569 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:20.441551    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:20.454084 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:20.454056    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:20.678140 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:20.678110    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-n46nx"
Jul 29 18:04:25.995239 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:04:25.995207    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:05:39.606407 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:05:39.606375    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:05:50.190123 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:05:50.190042    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:06:00.493279 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:06:00.493236    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:07:43.107371 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:07:43.107336    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:07:55.399177 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:07:55.399143    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:08:54.201909 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:08:54.201826    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:09:03.389175 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:09:03.389137    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:09:17.100741 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:09:17.100708    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:09:26.098854 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:09:26.098820    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:09:37.097224 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:09:37.097194    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:09:47.695093 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:09:47.695058    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:09:59.094323 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:09:59.094286    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:10:09.401527 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:10:09.401495    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:10:31.222347 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:10:31.222261    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:10:44.938007 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:10:44.937949    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:11:04.991916 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:11:04.991886    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:11:15.495736 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:11:15.495700    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:11:32.500598 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:11:32.500563    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:11:53.798256 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:11:53.798183    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:12:09.497774 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:12:09.497741    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:12:18.600079 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:12:18.600041    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:12:28.687973 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:12:28.687932    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:12:38.290081 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:12:38.290049    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:12:47.908233 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:12:47.908204    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:13:09.596292 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:13:09.596258    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:13:18.204363 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:13:18.204332    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:13:26.599121 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:13:26.599093    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:15:41.496038 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:15:41.495996    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:15:49.194678 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:15:49.194648    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:16:14.197109 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:16:14.197066    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:16:33.889916 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:16:33.889887    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:16:45.903320 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:16:45.903286    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:16:53.899211 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:16:53.899183    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:17:04.596733 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:17:04.596701    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:17:16.293917 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:17:16.293880    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:17:25.809285 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:17:25.809250    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:17:35.693842 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:17:35.693807    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:17:47.794976 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:17:47.794928    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:17:54.290808 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:17:54.290730    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:18:17.895761 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:17.895727    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:18:25.298381 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:25.298347    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:18:36.108778 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.108744    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"]
Jul 29 18:18:36.112496 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.112475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.114558 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.114539    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-8faee3e7-kserve-self-signed-certs\""
Jul 29 18:18:36.129895 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.129871    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"]
Jul 29 18:18:36.220510 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.220480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d393901d-b374-42fe-954a-acea74340f29-tls-certs\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.220656 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.220514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-home\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.220656 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.220532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-model-cache\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.220656 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.220612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-tmp-dir\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.220656 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.220636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khzmv\" (UniqueName: \"kubernetes.io/projected/d393901d-b374-42fe-954a-acea74340f29-kube-api-access-khzmv\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.220791 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.220669    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-dshm\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.321718 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.321684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-tmp-dir\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.321891 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.321741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khzmv\" (UniqueName: \"kubernetes.io/projected/d393901d-b374-42fe-954a-acea74340f29-kube-api-access-khzmv\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.321891 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.321808    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-dshm\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.321891 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.321834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d393901d-b374-42fe-954a-acea74340f29-tls-certs\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.321891 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.321853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-home\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.321891 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.321869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-model-cache\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.322260 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.322130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-tmp-dir\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.322326 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.322282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-model-cache\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.322473 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.322454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-home\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.324231 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.324209    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-dshm\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.325044 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.325019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d393901d-b374-42fe-954a-acea74340f29-tls-certs\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.331562 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.331538    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khzmv\" (UniqueName: \"kubernetes.io/projected/d393901d-b374-42fe-954a-acea74340f29-kube-api-access-khzmv\") pod \"e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") " pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.421549 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.421471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:36.557248 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.557217    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"]
Jul 29 18:18:36.560264 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:18:36.560231    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd393901d_b374_42fe_954a_acea74340f29.slice/crio-6299fbd5115293ca123ea8b536147590248b97911cf0450267275a4aa53ac70e WatchSource:0}: Error finding container 6299fbd5115293ca123ea8b536147590248b97911cf0450267275a4aa53ac70e: Status 404 returned error can't find the container with id 6299fbd5115293ca123ea8b536147590248b97911cf0450267275a4aa53ac70e
Jul 29 18:18:36.882183 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.882151    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2" event={"ID":"d393901d-b374-42fe-954a-acea74340f29","Type":"ContainerStarted","Data":"f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a"}
Jul 29 18:18:36.882183 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.882190    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2" event={"ID":"d393901d-b374-42fe-954a-acea74340f29","Type":"ContainerStarted","Data":"6299fbd5115293ca123ea8b536147590248b97911cf0450267275a4aa53ac70e"}
Jul 29 18:18:36.899631 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:36.899589    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2" podStartSLOduration=0.899572845 podStartE2EDuration="899.572845ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:18:36.897767446 +0000 UTC m=+1568.167266230" watchObservedRunningTime="2026-07-29 18:18:36.899572845 +0000 UTC m=+1568.169071606"
Jul 29 18:18:39.107220 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:39.107184    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:18:46.421925 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:46.421848    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:46.421925 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:46.421890    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:46.434160 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:46.434138    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:18:46.942034 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:18:46.942009    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:19:14.396844 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:19:14.396812    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:19:52.745441 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:19:52.745405    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"]
Jul 29 18:19:56.994973 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:19:56.994930    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:20:07.765386 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:07.765347    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2" podUID="d393901d-b374-42fe-954a-acea74340f29" containerName="main" containerID="cri-o://f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a" gracePeriod=45
Jul 29 18:20:08.016081 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.016029    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:20:08.134419 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134389    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-home\") pod \"d393901d-b374-42fe-954a-acea74340f29\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") "
Jul 29 18:20:08.134564 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134444    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khzmv\" (UniqueName: \"kubernetes.io/projected/d393901d-b374-42fe-954a-acea74340f29-kube-api-access-khzmv\") pod \"d393901d-b374-42fe-954a-acea74340f29\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") "
Jul 29 18:20:08.134564 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134481    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-tmp-dir\") pod \"d393901d-b374-42fe-954a-acea74340f29\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") "
Jul 29 18:20:08.134564 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134509    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d393901d-b374-42fe-954a-acea74340f29-tls-certs\") pod \"d393901d-b374-42fe-954a-acea74340f29\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") "
Jul 29 18:20:08.134564 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134537    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-model-cache\") pod \"d393901d-b374-42fe-954a-acea74340f29\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") "
Jul 29 18:20:08.134564 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134560    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-dshm\") pod \"d393901d-b374-42fe-954a-acea74340f29\" (UID: \"d393901d-b374-42fe-954a-acea74340f29\") "
Jul 29 18:20:08.134820 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134694    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-home" (OuterVolumeSpecName: "home") pod "d393901d-b374-42fe-954a-acea74340f29" (UID: "d393901d-b374-42fe-954a-acea74340f29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:20:08.134878 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134807    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d393901d-b374-42fe-954a-acea74340f29" (UID: "d393901d-b374-42fe-954a-acea74340f29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:20:08.134878 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134821    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-model-cache" (OuterVolumeSpecName: "model-cache") pod "d393901d-b374-42fe-954a-acea74340f29" (UID: "d393901d-b374-42fe-954a-acea74340f29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:20:08.134983 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134912    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-home\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:20:08.134983 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134935    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-tmp-dir\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:20:08.135054 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.134985    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-model-cache\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:20:08.136872 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.136848    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-dshm" (OuterVolumeSpecName: "dshm") pod "d393901d-b374-42fe-954a-acea74340f29" (UID: "d393901d-b374-42fe-954a-acea74340f29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:20:08.136984 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.136888    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d393901d-b374-42fe-954a-acea74340f29-kube-api-access-khzmv" (OuterVolumeSpecName: "kube-api-access-khzmv") pod "d393901d-b374-42fe-954a-acea74340f29" (UID: "d393901d-b374-42fe-954a-acea74340f29"). InnerVolumeSpecName "kube-api-access-khzmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:20:08.136984 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.136902    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d393901d-b374-42fe-954a-acea74340f29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d393901d-b374-42fe-954a-acea74340f29" (UID: "d393901d-b374-42fe-954a-acea74340f29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:20:08.235575 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.235547    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khzmv\" (UniqueName: \"kubernetes.io/projected/d393901d-b374-42fe-954a-acea74340f29-kube-api-access-khzmv\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:20:08.235575 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.235576    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d393901d-b374-42fe-954a-acea74340f29-tls-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:20:08.235721 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.235585    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d393901d-b374-42fe-954a-acea74340f29-dshm\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:20:08.338380 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.338356    2580 generic.go:356] "Generic (PLEG): container finished" podID="d393901d-b374-42fe-954a-acea74340f29" containerID="f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a" exitCode=0
Jul 29 18:20:08.338544 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.338403    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"
Jul 29 18:20:08.338544 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.338425    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2" event={"ID":"d393901d-b374-42fe-954a-acea74340f29","Type":"ContainerDied","Data":"f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a"}
Jul 29 18:20:08.338544 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.338453    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2" event={"ID":"d393901d-b374-42fe-954a-acea74340f29","Type":"ContainerDied","Data":"6299fbd5115293ca123ea8b536147590248b97911cf0450267275a4aa53ac70e"}
Jul 29 18:20:08.338544 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.338470    2580 scope.go:117] "RemoveContainer" containerID="f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a"
Jul 29 18:20:08.347860 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.347843    2580 scope.go:117] "RemoveContainer" containerID="f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a"
Jul 29 18:20:08.348137 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:20:08.348119    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a\": container with ID starting with f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a not found: ID does not exist" containerID="f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a"
Jul 29 18:20:08.348198 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.348144    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a"} err="failed to get container status \"f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a\": rpc error: code = NotFound desc = could not find container \"f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a\": container with ID starting with f08c8ee791e7ab522edc80dde22c430703e2ab61a7570f546107a85c260d049a not found: ID does not exist"
Jul 29 18:20:08.357577 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.357553    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"]
Jul 29 18:20:08.359875 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:08.359856    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-8faee3e7-kserve-678d88bf9-zmfd2"]
Jul 29 18:20:09.272559 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:09.272527    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d393901d-b374-42fe-954a-acea74340f29" path="/var/lib/kubelet/pods/d393901d-b374-42fe-954a-acea74340f29/volumes"
Jul 29 18:20:10.794085 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:10.794048    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:20:17.990660 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:17.990625    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:20:27.101774 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:27.101737    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:20:34.097642 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:34.097601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:20:47.900510 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:47.900476    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:20:57.290205 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:20:57.290125    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:21:08.996936 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:21:08.996902    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:21:18.204624 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:21:18.204590    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:21:27.291385 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:21:27.291354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:21:40.095866 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:21:40.095832    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:21:48.805199 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:21:48.805162    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:21:58.197908 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:21:58.197874    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:22:09.199625 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:22:09.199596    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:22:21.894344 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:22:21.894262    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:22:31.496823 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:22:31.496790    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:22:53.201971 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:22:53.201933    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:23:05.399338 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:23:05.399304    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:23:21.194193 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:23:21.194156    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:23:35.994123 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:23:35.994084    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:23:45.494887 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:23:45.494842    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:23:59.293634 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:23:59.293544    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:24:12.896473 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:12.896442    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:24:22.805547 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:22.805508    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:24:33.701020 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:33.700982    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:24:38.289850 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.289804    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"]
Jul 29 18:24:38.290272 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.290258    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d393901d-b374-42fe-954a-acea74340f29" containerName="main"
Jul 29 18:24:38.290315 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.290274    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d393901d-b374-42fe-954a-acea74340f29" containerName="main"
Jul 29 18:24:38.290359 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.290351    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d393901d-b374-42fe-954a-acea74340f29" containerName="main"
Jul 29 18:24:38.293354 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.293337    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:24:38.295288 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.295267    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 18:24:38.295832 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.295812    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0174a53b\""
Jul 29 18:24:38.295832 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.295822    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0174a53b-dockercfg-pmkg4\""
Jul 29 18:24:38.296026 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.295859    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 18:24:38.300426 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.300399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz9bc\" (UniqueName: \"kubernetes.io/projected/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-kube-api-access-gz9bc\") pod \"payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7\" (UID: \"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:24:38.300554 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.300452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7\" (UID: \"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:24:38.300554 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.300521    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"]
Jul 29 18:24:38.330193 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.330165    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"]
Jul 29 18:24:38.333449 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.333432    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:24:38.343424 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.343402    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"]
Jul 29 18:24:38.401274 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.401242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz9bc\" (UniqueName: \"kubernetes.io/projected/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-kube-api-access-gz9bc\") pod \"payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7\" (UID: \"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:24:38.401444 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.401280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7\" (UID: \"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:24:38.401444 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.401312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c2fr\" (UniqueName: \"kubernetes.io/projected/56acc7f1-e012-4a47-8c4b-292a4ca7de69-kube-api-access-5c2fr\") pod \"payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht\" (UID: \"56acc7f1-e012-4a47-8c4b-292a4ca7de69\") " pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:24:38.401444 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.401434    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56acc7f1-e012-4a47-8c4b-292a4ca7de69-plugins-config-volume\") pod \"payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht\" (UID: \"56acc7f1-e012-4a47-8c4b-292a4ca7de69\") " pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:24:38.401982 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.401940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7\" (UID: \"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:24:38.409117 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.409096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz9bc\" (UniqueName: \"kubernetes.io/projected/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-kube-api-access-gz9bc\") pod \"payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7\" (UID: \"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:24:38.501968 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.501934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c2fr\" (UniqueName: \"kubernetes.io/projected/56acc7f1-e012-4a47-8c4b-292a4ca7de69-kube-api-access-5c2fr\") pod \"payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht\" (UID: \"56acc7f1-e012-4a47-8c4b-292a4ca7de69\") " pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:24:38.502101 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.501991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56acc7f1-e012-4a47-8c4b-292a4ca7de69-plugins-config-volume\") pod \"payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht\" (UID: \"56acc7f1-e012-4a47-8c4b-292a4ca7de69\") " pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:24:38.502688 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.502670    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56acc7f1-e012-4a47-8c4b-292a4ca7de69-plugins-config-volume\") pod \"payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht\" (UID: \"56acc7f1-e012-4a47-8c4b-292a4ca7de69\") " pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:24:38.510312 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.510287    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c2fr\" (UniqueName: \"kubernetes.io/projected/56acc7f1-e012-4a47-8c4b-292a4ca7de69-kube-api-access-5c2fr\") pod \"payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht\" (UID: \"56acc7f1-e012-4a47-8c4b-292a4ca7de69\") " pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:24:38.605314 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.605246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:24:38.644786 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.644761    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:24:38.758782 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.758752    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"]
Jul 29 18:24:38.761066 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:24:38.761019    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11818eb0_5f72_4a99_98ff_9a1c4c7ceb16.slice/crio-22d54e11d94dc6b40f85d1c09f4247a590c372a2574ded00a78d0d0e94122c1e WatchSource:0}: Error finding container 22d54e11d94dc6b40f85d1c09f4247a590c372a2574ded00a78d0d0e94122c1e: Status 404 returned error can't find the container with id 22d54e11d94dc6b40f85d1c09f4247a590c372a2574ded00a78d0d0e94122c1e
Jul 29 18:24:38.763414 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.763397    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 18:24:38.798627 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:38.798605    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"]
Jul 29 18:24:38.800099 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:24:38.800073    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56acc7f1_e012_4a47_8c4b_292a4ca7de69.slice/crio-697e8ceefd231ef47a97dc2c09ec3d1434c3336827481c494dc0816d9283361c WatchSource:0}: Error finding container 697e8ceefd231ef47a97dc2c09ec3d1434c3336827481c494dc0816d9283361c: Status 404 returned error can't find the container with id 697e8ceefd231ef47a97dc2c09ec3d1434c3336827481c494dc0816d9283361c
Jul 29 18:24:39.676773 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:39.676730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7" event={"ID":"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16","Type":"ContainerStarted","Data":"22d54e11d94dc6b40f85d1c09f4247a590c372a2574ded00a78d0d0e94122c1e"}
Jul 29 18:24:39.679431 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:39.679356    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht" event={"ID":"56acc7f1-e012-4a47-8c4b-292a4ca7de69","Type":"ContainerStarted","Data":"697e8ceefd231ef47a97dc2c09ec3d1434c3336827481c494dc0816d9283361c"}
Jul 29 18:24:41.694683 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:41.694644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7" event={"ID":"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16","Type":"ContainerStarted","Data":"3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3"}
Jul 29 18:24:41.695134 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:41.694777    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:24:41.696233 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:41.696209    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht" event={"ID":"56acc7f1-e012-4a47-8c4b-292a4ca7de69","Type":"ContainerStarted","Data":"62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c"}
Jul 29 18:24:41.696361 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:41.696347    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:24:41.709658 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:41.709604    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7" podStartSLOduration=1.851887139 podStartE2EDuration="3.709588249s" totalImagesPullingTime="1.85770111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:24:41.707473386 +0000 UTC m=+1932.976972172" watchObservedRunningTime="2026-07-29 18:24:41.709588249 +0000 UTC m=+1932.979087032"
Jul 29 18:24:41.723217 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:41.723174    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht" podStartSLOduration=1.89756015 podStartE2EDuration="3.723160881s" totalImagesPullingTime="1.825600731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:24:41.720775467 +0000 UTC m=+1932.990274260" watchObservedRunningTime="2026-07-29 18:24:41.723160881 +0000 UTC m=+1932.992659665"
Jul 29 18:24:44.793482 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:44.793446    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:24:52.704091 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:52.704065    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:24:52.705834 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:24:52.704132    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:25:18.437800 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.437768    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"]
Jul 29 18:25:18.438369 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.437990    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht" podUID="56acc7f1-e012-4a47-8c4b-292a4ca7de69" containerName="payload-processing" containerID="cri-o://62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c" gracePeriod=30
Jul 29 18:25:18.445485 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.445457    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"]
Jul 29 18:25:18.445767 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.445732    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7" podUID="11818eb0-5f72-4a99-98ff-9a1c4c7ceb16" containerName="payload-pre-processing" containerID="cri-o://3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3" gracePeriod=30
Jul 29 18:25:18.731430 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.731401    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:25:18.737522 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.737500    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:25:18.777812 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.777781    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-plugins-config-volume\") pod \"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16\" (UID: \"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16\") "
Jul 29 18:25:18.777999 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.777843    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gz9bc\" (UniqueName: \"kubernetes.io/projected/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-kube-api-access-gz9bc\") pod \"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16\" (UID: \"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16\") "
Jul 29 18:25:18.777999 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.777912    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5c2fr\" (UniqueName: \"kubernetes.io/projected/56acc7f1-e012-4a47-8c4b-292a4ca7de69-kube-api-access-5c2fr\") pod \"56acc7f1-e012-4a47-8c4b-292a4ca7de69\" (UID: \"56acc7f1-e012-4a47-8c4b-292a4ca7de69\") "
Jul 29 18:25:18.777999 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.777943    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56acc7f1-e012-4a47-8c4b-292a4ca7de69-plugins-config-volume\") pod \"56acc7f1-e012-4a47-8c4b-292a4ca7de69\" (UID: \"56acc7f1-e012-4a47-8c4b-292a4ca7de69\") "
Jul 29 18:25:18.778263 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.778238    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11818eb0-5f72-4a99-98ff-9a1c4c7ceb16" (UID: "11818eb0-5f72-4a99-98ff-9a1c4c7ceb16"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:25:18.778429 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.778399    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56acc7f1-e012-4a47-8c4b-292a4ca7de69-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56acc7f1-e012-4a47-8c4b-292a4ca7de69" (UID: "56acc7f1-e012-4a47-8c4b-292a4ca7de69"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:25:18.780315 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.780285    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56acc7f1-e012-4a47-8c4b-292a4ca7de69-kube-api-access-5c2fr" (OuterVolumeSpecName: "kube-api-access-5c2fr") pod "56acc7f1-e012-4a47-8c4b-292a4ca7de69" (UID: "56acc7f1-e012-4a47-8c4b-292a4ca7de69"). InnerVolumeSpecName "kube-api-access-5c2fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:25:18.780651 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.780630    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-kube-api-access-gz9bc" (OuterVolumeSpecName: "kube-api-access-gz9bc") pod "11818eb0-5f72-4a99-98ff-9a1c4c7ceb16" (UID: "11818eb0-5f72-4a99-98ff-9a1c4c7ceb16"). InnerVolumeSpecName "kube-api-access-gz9bc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:25:18.878905 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.878873    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz9bc\" (UniqueName: \"kubernetes.io/projected/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-kube-api-access-gz9bc\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:25:18.878905 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.878900    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c2fr\" (UniqueName: \"kubernetes.io/projected/56acc7f1-e012-4a47-8c4b-292a4ca7de69-kube-api-access-5c2fr\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:25:18.878905 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.878909    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56acc7f1-e012-4a47-8c4b-292a4ca7de69-plugins-config-volume\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:25:18.879133 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.878919    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16-plugins-config-volume\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:25:18.887097 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.887069    2580 generic.go:356] "Generic (PLEG): container finished" podID="11818eb0-5f72-4a99-98ff-9a1c4c7ceb16" containerID="3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3" exitCode=0
Jul 29 18:25:18.887225 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.887105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7" event={"ID":"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16","Type":"ContainerDied","Data":"3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3"}
Jul 29 18:25:18.887225 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.887134    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"
Jul 29 18:25:18.887225 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.887184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7" event={"ID":"11818eb0-5f72-4a99-98ff-9a1c4c7ceb16","Type":"ContainerDied","Data":"22d54e11d94dc6b40f85d1c09f4247a590c372a2574ded00a78d0d0e94122c1e"}
Jul 29 18:25:18.887225 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.887204    2580 scope.go:117] "RemoveContainer" containerID="3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3"
Jul 29 18:25:18.888562 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.888538    2580 generic.go:356] "Generic (PLEG): container finished" podID="56acc7f1-e012-4a47-8c4b-292a4ca7de69" containerID="62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c" exitCode=0
Jul 29 18:25:18.888667 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.888590    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"
Jul 29 18:25:18.888667 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.888593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht" event={"ID":"56acc7f1-e012-4a47-8c4b-292a4ca7de69","Type":"ContainerDied","Data":"62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c"}
Jul 29 18:25:18.888667 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.888631    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht" event={"ID":"56acc7f1-e012-4a47-8c4b-292a4ca7de69","Type":"ContainerDied","Data":"697e8ceefd231ef47a97dc2c09ec3d1434c3336827481c494dc0816d9283361c"}
Jul 29 18:25:18.897354 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.897335    2580 scope.go:117] "RemoveContainer" containerID="3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3"
Jul 29 18:25:18.897638 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:25:18.897615    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3\": container with ID starting with 3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3 not found: ID does not exist" containerID="3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3"
Jul 29 18:25:18.897705 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.897647    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3"} err="failed to get container status \"3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3\": rpc error: code = NotFound desc = could not find container \"3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3\": container with ID starting with 3060abe69ed83cc3a1f0e8daab548462592413d0af818fc38f8f79de3a3ab5c3 not found: ID does not exist"
Jul 29 18:25:18.897705 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.897667    2580 scope.go:117] "RemoveContainer" containerID="62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c"
Jul 29 18:25:18.906201 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.906180    2580 scope.go:117] "RemoveContainer" containerID="62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c"
Jul 29 18:25:18.906430 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:25:18.906413    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c\": container with ID starting with 62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c not found: ID does not exist" containerID="62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c"
Jul 29 18:25:18.906473 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.906437    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c"} err="failed to get container status \"62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c\": rpc error: code = NotFound desc = could not find container \"62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c\": container with ID starting with 62cd08252fd22a546ca6d401222012d4cfc3217372d1476660514a2eb1c3195c not found: ID does not exist"
Jul 29 18:25:18.912687 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.912669    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"]
Jul 29 18:25:18.915354 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.915332    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0174a53b-6756d4c4f8-qvzht"]
Jul 29 18:25:18.924306 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.924200    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"]
Jul 29 18:25:18.925780 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:18.925763    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0174a53b-59754578c6-h9br7"]
Jul 29 18:25:19.272982 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:19.272932    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11818eb0-5f72-4a99-98ff-9a1c4c7ceb16" path="/var/lib/kubelet/pods/11818eb0-5f72-4a99-98ff-9a1c4c7ceb16/volumes"
Jul 29 18:25:19.273324 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:19.273311    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56acc7f1-e012-4a47-8c4b-292a4ca7de69" path="/var/lib/kubelet/pods/56acc7f1-e012-4a47-8c4b-292a4ca7de69/volumes"
Jul 29 18:25:22.491885 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.491813    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:25:22.761521 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.761450    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"]
Jul 29 18:25:22.761845 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.761833    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11818eb0-5f72-4a99-98ff-9a1c4c7ceb16" containerName="payload-pre-processing"
Jul 29 18:25:22.761887 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.761846    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="11818eb0-5f72-4a99-98ff-9a1c4c7ceb16" containerName="payload-pre-processing"
Jul 29 18:25:22.761887 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.761855    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56acc7f1-e012-4a47-8c4b-292a4ca7de69" containerName="payload-processing"
Jul 29 18:25:22.761887 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.761860    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="56acc7f1-e012-4a47-8c4b-292a4ca7de69" containerName="payload-processing"
Jul 29 18:25:22.762042 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.761932    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="11818eb0-5f72-4a99-98ff-9a1c4c7ceb16" containerName="payload-pre-processing"
Jul 29 18:25:22.762042 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.761943    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="56acc7f1-e012-4a47-8c4b-292a4ca7de69" containerName="payload-processing"
Jul 29 18:25:22.765776 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.765731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:22.768671 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.768649    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 18:25:22.768786 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.768744    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 18:25:22.768786 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.768744    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-fa2af080-dockercfg-ldpqf\""
Jul 29 18:25:22.769131 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.769115    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-fa2af080\""
Jul 29 18:25:22.778153 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.778133    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"]
Jul 29 18:25:22.779849 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.779829    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"]
Jul 29 18:25:22.782641 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.782614    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:22.798740 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.798713    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"]
Jul 29 18:25:22.811277 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.811253    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brqsv\" (UniqueName: \"kubernetes.io/projected/496f4114-2dc0-497e-a41a-d499b711a151-kube-api-access-brqsv\") pod \"payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k\" (UID: \"496f4114-2dc0-497e-a41a-d499b711a151\") " pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:22.811420 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.811290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzhdp\" (UniqueName: \"kubernetes.io/projected/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-kube-api-access-qzhdp\") pod \"payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g\" (UID: \"69873a53-35f3-490f-a6e2-e10ab3a5e6f6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:22.811420 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.811316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g\" (UID: \"69873a53-35f3-490f-a6e2-e10ab3a5e6f6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:22.811420 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.811389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/496f4114-2dc0-497e-a41a-d499b711a151-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k\" (UID: \"496f4114-2dc0-497e-a41a-d499b711a151\") " pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:22.912011 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.911969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/496f4114-2dc0-497e-a41a-d499b711a151-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k\" (UID: \"496f4114-2dc0-497e-a41a-d499b711a151\") " pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:22.912171 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.912071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brqsv\" (UniqueName: \"kubernetes.io/projected/496f4114-2dc0-497e-a41a-d499b711a151-kube-api-access-brqsv\") pod \"payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k\" (UID: \"496f4114-2dc0-497e-a41a-d499b711a151\") " pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:22.912171 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.912101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzhdp\" (UniqueName: \"kubernetes.io/projected/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-kube-api-access-qzhdp\") pod \"payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g\" (UID: \"69873a53-35f3-490f-a6e2-e10ab3a5e6f6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:22.912171 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.912125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g\" (UID: \"69873a53-35f3-490f-a6e2-e10ab3a5e6f6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:22.912704 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.912685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/496f4114-2dc0-497e-a41a-d499b711a151-plugins-config-volume\") pod \"payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k\" (UID: \"496f4114-2dc0-497e-a41a-d499b711a151\") " pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:22.912782 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.912765    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g\" (UID: \"69873a53-35f3-490f-a6e2-e10ab3a5e6f6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:22.919689 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.919668    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzhdp\" (UniqueName: \"kubernetes.io/projected/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-kube-api-access-qzhdp\") pod \"payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g\" (UID: \"69873a53-35f3-490f-a6e2-e10ab3a5e6f6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:22.919832 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:22.919815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brqsv\" (UniqueName: \"kubernetes.io/projected/496f4114-2dc0-497e-a41a-d499b711a151-kube-api-access-brqsv\") pod \"payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k\" (UID: \"496f4114-2dc0-497e-a41a-d499b711a151\") " pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:23.077615 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.077586    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:23.093470 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.093445    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:23.249031 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.248978    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"]
Jul 29 18:25:23.251353 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:25:23.251325    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod496f4114_2dc0_497e_a41a_d499b711a151.slice/crio-19d93c0d968b1874ebc724689c109ced540b852b2afb3da23c8b78760a7ff93a WatchSource:0}: Error finding container 19d93c0d968b1874ebc724689c109ced540b852b2afb3da23c8b78760a7ff93a: Status 404 returned error can't find the container with id 19d93c0d968b1874ebc724689c109ced540b852b2afb3da23c8b78760a7ff93a
Jul 29 18:25:23.420589 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.420427    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"]
Jul 29 18:25:23.423203 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:25:23.423177    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69873a53_35f3_490f_a6e2_e10ab3a5e6f6.slice/crio-6dcbd79ff4daa218dcef768d98c2bf4ee4008506fa2fe282009f7350336f80bc WatchSource:0}: Error finding container 6dcbd79ff4daa218dcef768d98c2bf4ee4008506fa2fe282009f7350336f80bc: Status 404 returned error can't find the container with id 6dcbd79ff4daa218dcef768d98c2bf4ee4008506fa2fe282009f7350336f80bc
Jul 29 18:25:23.916659 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.916623    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k" event={"ID":"496f4114-2dc0-497e-a41a-d499b711a151","Type":"ContainerStarted","Data":"2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17"}
Jul 29 18:25:23.917212 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.916691    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:23.917212 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.916713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k" event={"ID":"496f4114-2dc0-497e-a41a-d499b711a151","Type":"ContainerStarted","Data":"19d93c0d968b1874ebc724689c109ced540b852b2afb3da23c8b78760a7ff93a"}
Jul 29 18:25:23.918256 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.918232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g" event={"ID":"69873a53-35f3-490f-a6e2-e10ab3a5e6f6","Type":"ContainerStarted","Data":"267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd"}
Jul 29 18:25:23.918377 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.918294    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g" event={"ID":"69873a53-35f3-490f-a6e2-e10ab3a5e6f6","Type":"ContainerStarted","Data":"6dcbd79ff4daa218dcef768d98c2bf4ee4008506fa2fe282009f7350336f80bc"}
Jul 29 18:25:23.918377 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.918323    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:23.931332 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.931293    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k" podStartSLOduration=1.9312789609999998 podStartE2EDuration="1.931278961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:25:23.930186611 +0000 UTC m=+1975.199685414" watchObservedRunningTime="2026-07-29 18:25:23.931278961 +0000 UTC m=+1975.200777794"
Jul 29 18:25:23.945611 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:23.945563    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g" podStartSLOduration=1.945549931 podStartE2EDuration="1.945549931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:25:23.943803975 +0000 UTC m=+1975.213302768" watchObservedRunningTime="2026-07-29 18:25:23.945549931 +0000 UTC m=+1975.215048714"
Jul 29 18:25:33.796149 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:33.796109    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:25:34.925160 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:34.925120    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:34.925521 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:34.925185    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:52.614096 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:52.614062    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"]
Jul 29 18:25:52.614538 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:52.614313    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k" podUID="496f4114-2dc0-497e-a41a-d499b711a151" containerName="payload-processing" containerID="cri-o://2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17" gracePeriod=30
Jul 29 18:25:52.618861 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:52.618835    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"]
Jul 29 18:25:52.619269 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:52.619241    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g" podUID="69873a53-35f3-490f-a6e2-e10ab3a5e6f6" containerName="payload-pre-processing" containerID="cri-o://267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd" gracePeriod=30
Jul 29 18:25:52.902027 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:52.902001    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:52.912893 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:52.912869    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:53.066374 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.066340    2580 generic.go:356] "Generic (PLEG): container finished" podID="496f4114-2dc0-497e-a41a-d499b711a151" containerID="2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17" exitCode=0
Jul 29 18:25:53.066552 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.066392    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"
Jul 29 18:25:53.066552 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.066429    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k" event={"ID":"496f4114-2dc0-497e-a41a-d499b711a151","Type":"ContainerDied","Data":"2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17"}
Jul 29 18:25:53.066552 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.066499    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k" event={"ID":"496f4114-2dc0-497e-a41a-d499b711a151","Type":"ContainerDied","Data":"19d93c0d968b1874ebc724689c109ced540b852b2afb3da23c8b78760a7ff93a"}
Jul 29 18:25:53.066552 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.066533    2580 scope.go:117] "RemoveContainer" containerID="2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17"
Jul 29 18:25:53.067975 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.067934    2580 generic.go:356] "Generic (PLEG): container finished" podID="69873a53-35f3-490f-a6e2-e10ab3a5e6f6" containerID="267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd" exitCode=0
Jul 29 18:25:53.068090 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.067998    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"
Jul 29 18:25:53.068090 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.067998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g" event={"ID":"69873a53-35f3-490f-a6e2-e10ab3a5e6f6","Type":"ContainerDied","Data":"267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd"}
Jul 29 18:25:53.068090 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.068047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g" event={"ID":"69873a53-35f3-490f-a6e2-e10ab3a5e6f6","Type":"ContainerDied","Data":"6dcbd79ff4daa218dcef768d98c2bf4ee4008506fa2fe282009f7350336f80bc"}
Jul 29 18:25:53.070477 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.070456    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/496f4114-2dc0-497e-a41a-d499b711a151-plugins-config-volume\") pod \"496f4114-2dc0-497e-a41a-d499b711a151\" (UID: \"496f4114-2dc0-497e-a41a-d499b711a151\") "
Jul 29 18:25:53.070606 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.070520    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzhdp\" (UniqueName: \"kubernetes.io/projected/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-kube-api-access-qzhdp\") pod \"69873a53-35f3-490f-a6e2-e10ab3a5e6f6\" (UID: \"69873a53-35f3-490f-a6e2-e10ab3a5e6f6\") "
Jul 29 18:25:53.070606 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.070547    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brqsv\" (UniqueName: \"kubernetes.io/projected/496f4114-2dc0-497e-a41a-d499b711a151-kube-api-access-brqsv\") pod \"496f4114-2dc0-497e-a41a-d499b711a151\" (UID: \"496f4114-2dc0-497e-a41a-d499b711a151\") "
Jul 29 18:25:53.070606 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.070579    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-plugins-config-volume\") pod \"69873a53-35f3-490f-a6e2-e10ab3a5e6f6\" (UID: \"69873a53-35f3-490f-a6e2-e10ab3a5e6f6\") "
Jul 29 18:25:53.070911 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.070884    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/496f4114-2dc0-497e-a41a-d499b711a151-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "496f4114-2dc0-497e-a41a-d499b711a151" (UID: "496f4114-2dc0-497e-a41a-d499b711a151"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:25:53.071047 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.071025    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69873a53-35f3-490f-a6e2-e10ab3a5e6f6" (UID: "69873a53-35f3-490f-a6e2-e10ab3a5e6f6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:25:53.072788 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.072765    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/496f4114-2dc0-497e-a41a-d499b711a151-kube-api-access-brqsv" (OuterVolumeSpecName: "kube-api-access-brqsv") pod "496f4114-2dc0-497e-a41a-d499b711a151" (UID: "496f4114-2dc0-497e-a41a-d499b711a151"). InnerVolumeSpecName "kube-api-access-brqsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:25:53.073043 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.073012    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-kube-api-access-qzhdp" (OuterVolumeSpecName: "kube-api-access-qzhdp") pod "69873a53-35f3-490f-a6e2-e10ab3a5e6f6" (UID: "69873a53-35f3-490f-a6e2-e10ab3a5e6f6"). InnerVolumeSpecName "kube-api-access-qzhdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:25:53.076298 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.076282    2580 scope.go:117] "RemoveContainer" containerID="2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17"
Jul 29 18:25:53.076550 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:25:53.076529    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17\": container with ID starting with 2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17 not found: ID does not exist" containerID="2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17"
Jul 29 18:25:53.076617 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.076562    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17"} err="failed to get container status \"2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17\": rpc error: code = NotFound desc = could not find container \"2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17\": container with ID starting with 2fcc1b4476d900bb8e481f21814cc1abac06de7287db78a5ea6cf7bb3f2bfd17 not found: ID does not exist"
Jul 29 18:25:53.076617 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.076590    2580 scope.go:117] "RemoveContainer" containerID="267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd"
Jul 29 18:25:53.084003 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.083987    2580 scope.go:117] "RemoveContainer" containerID="267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd"
Jul 29 18:25:53.084254 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:25:53.084231    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd\": container with ID starting with 267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd not found: ID does not exist" containerID="267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd"
Jul 29 18:25:53.084308 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.084252    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd"} err="failed to get container status \"267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd\": rpc error: code = NotFound desc = could not find container \"267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd\": container with ID starting with 267713e322f815a9e9c77805a75ed93cbc4c40f061aa5b3cb1967760030f5dcd not found: ID does not exist"
Jul 29 18:25:53.171434 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.171385    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/496f4114-2dc0-497e-a41a-d499b711a151-plugins-config-volume\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:25:53.171434 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.171405    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzhdp\" (UniqueName: \"kubernetes.io/projected/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-kube-api-access-qzhdp\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:25:53.171434 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.171415    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brqsv\" (UniqueName: \"kubernetes.io/projected/496f4114-2dc0-497e-a41a-d499b711a151-kube-api-access-brqsv\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:25:53.171434 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.171424    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69873a53-35f3-490f-a6e2-e10ab3a5e6f6-plugins-config-volume\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:25:53.384801 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.384771    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"]
Jul 29 18:25:53.388881 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.388858    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-fa2af080-6c496888f7-f7x5k"]
Jul 29 18:25:53.398208 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.398188    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"]
Jul 29 18:25:53.403748 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.403729    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-fa2af080-7cbc4844-g9q2g"]
Jul 29 18:25:53.665299 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.665268    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"]
Jul 29 18:25:53.666682 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.666655    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="496f4114-2dc0-497e-a41a-d499b711a151" containerName="payload-processing"
Jul 29 18:25:53.666845 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.666834    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="496f4114-2dc0-497e-a41a-d499b711a151" containerName="payload-processing"
Jul 29 18:25:53.666930 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.666920    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69873a53-35f3-490f-a6e2-e10ab3a5e6f6" containerName="payload-pre-processing"
Jul 29 18:25:53.667192 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.667177    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="69873a53-35f3-490f-a6e2-e10ab3a5e6f6" containerName="payload-pre-processing"
Jul 29 18:25:53.667611 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.667596    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="496f4114-2dc0-497e-a41a-d499b711a151" containerName="payload-processing"
Jul 29 18:25:53.667767 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.667753    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="69873a53-35f3-490f-a6e2-e10ab3a5e6f6" containerName="payload-pre-processing"
Jul 29 18:25:53.673694 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.673674    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.676351 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.676328    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 18:25:53.676457 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.676333    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 18:25:53.676886 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.676845    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 18:25:53.677196 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.677152    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f861467d-openshift-default-dockercfg-h2kgq\""
Jul 29 18:25:53.683339 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.683320    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"]
Jul 29 18:25:53.776148 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.776119    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-istio-token\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.776274 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.776160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-certs\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.776274 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.776185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-data\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.776274 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.776249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpmm7\" (UniqueName: \"kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-kube-api-access-cpmm7\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.776401 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.776277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-envoy\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.776401 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.776305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-socket\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.776401 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.776332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/912b93b1-1765-4492-8b95-0efd03039d99-istiod-ca-cert\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.776512 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.776406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-credential-socket\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.776512 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.776430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/912b93b1-1765-4492-8b95-0efd03039d99-istio-podinfo\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877184 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/912b93b1-1765-4492-8b95-0efd03039d99-istio-podinfo\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877285 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-istio-token\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877285 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-certs\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877285 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-data\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpmm7\" (UniqueName: \"kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-kube-api-access-cpmm7\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877511 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-envoy\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877577 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-socket\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877577 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/912b93b1-1765-4492-8b95-0efd03039d99-istiod-ca-cert\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877577 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-data\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877747 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-credential-socket\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877747 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877620    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-certs\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.877859 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877843    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-credential-socket\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.878114 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.877990    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-socket\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.878164 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.878154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/912b93b1-1765-4492-8b95-0efd03039d99-istiod-ca-cert\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.879762 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.879736    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-envoy\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.879929 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.879914    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/912b93b1-1765-4492-8b95-0efd03039d99-istio-podinfo\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.886934 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.886871    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-istio-token\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.887709 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.887688    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpmm7\" (UniqueName: \"kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-kube-api-access-cpmm7\") pod \"e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") " pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:53.987014 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:53.986990    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:54.126289 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:54.126261    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"]
Jul 29 18:25:54.128374 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:25:54.128344    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod912b93b1_1765_4492_8b95_0efd03039d99.slice/crio-a325f0ec14bb280c34ed9e4c18f73a6280416ddb737939d87e29e9c35068a520 WatchSource:0}: Error finding container a325f0ec14bb280c34ed9e4c18f73a6280416ddb737939d87e29e9c35068a520: Status 404 returned error can't find the container with id a325f0ec14bb280c34ed9e4c18f73a6280416ddb737939d87e29e9c35068a520
Jul 29 18:25:55.082622 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:55.082589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" event={"ID":"912b93b1-1765-4492-8b95-0efd03039d99","Type":"ContainerStarted","Data":"a325f0ec14bb280c34ed9e4c18f73a6280416ddb737939d87e29e9c35068a520"}
Jul 29 18:25:55.274699 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:55.274669    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="496f4114-2dc0-497e-a41a-d499b711a151" path="/var/lib/kubelet/pods/496f4114-2dc0-497e-a41a-d499b711a151/volumes"
Jul 29 18:25:55.275076 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:55.275061    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69873a53-35f3-490f-a6e2-e10ab3a5e6f6" path="/var/lib/kubelet/pods/69873a53-35f3-490f-a6e2-e10ab3a5e6f6/volumes"
Jul 29 18:25:56.297571 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:56.297537    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:25:57.012929 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:57.012889    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:25:57.013066 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:57.012980    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:25:57.013066 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:57.013010    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:25:57.096647 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:57.096593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" event={"ID":"912b93b1-1765-4492-8b95-0efd03039d99","Type":"ContainerStarted","Data":"6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb"}
Jul 29 18:25:57.120380 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:57.120332    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" podStartSLOduration=1.238010077 podStartE2EDuration="4.120316804s" totalImagesPullingTime="2.882306727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:25:57.118265586 +0000 UTC m=+2008.387764403" watchObservedRunningTime="2026-07-29 18:25:57.120316804 +0000 UTC m=+2008.389815661"
Jul 29 18:25:57.987195 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:57.987170    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:58.992028 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:58.991995    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:59.112201 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.112172    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:59.112863 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.112843    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:25:59.423763 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.423725    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"]
Jul 29 18:25:59.427505 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.427482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:25:59.429847 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.429826    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f861467d-dockercfg-nhrcg\""
Jul 29 18:25:59.429995 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.429880    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f861467d\""
Jul 29 18:25:59.443073 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.443050    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"]
Jul 29 18:25:59.530023 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.529994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgmm5\" (UniqueName: \"kubernetes.io/projected/e4813cc7-8519-4636-b64f-1832de893f82-kube-api-access-lgmm5\") pod \"payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk\" (UID: \"e4813cc7-8519-4636-b64f-1832de893f82\") " pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:25:59.530183 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.530040    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4813cc7-8519-4636-b64f-1832de893f82-plugins-config-volume\") pod \"payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk\" (UID: \"e4813cc7-8519-4636-b64f-1832de893f82\") " pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:25:59.631430 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.631392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgmm5\" (UniqueName: \"kubernetes.io/projected/e4813cc7-8519-4636-b64f-1832de893f82-kube-api-access-lgmm5\") pod \"payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk\" (UID: \"e4813cc7-8519-4636-b64f-1832de893f82\") " pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:25:59.631593 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.631445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4813cc7-8519-4636-b64f-1832de893f82-plugins-config-volume\") pod \"payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk\" (UID: \"e4813cc7-8519-4636-b64f-1832de893f82\") " pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:25:59.632141 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.632116    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4813cc7-8519-4636-b64f-1832de893f82-plugins-config-volume\") pod \"payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk\" (UID: \"e4813cc7-8519-4636-b64f-1832de893f82\") " pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:25:59.642128 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.642101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgmm5\" (UniqueName: \"kubernetes.io/projected/e4813cc7-8519-4636-b64f-1832de893f82-kube-api-access-lgmm5\") pod \"payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk\" (UID: \"e4813cc7-8519-4636-b64f-1832de893f82\") " pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:25:59.738319 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:25:59.738236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:26:00.096891 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:00.096857    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"]
Jul 29 18:26:00.104302 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:26:00.104272    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4813cc7_8519_4636_b64f_1832de893f82.slice/crio-16c08191db3ba8aa73490757d8929d5323fa14c6cdcd0a3fc8f0186f42c9100d WatchSource:0}: Error finding container 16c08191db3ba8aa73490757d8929d5323fa14c6cdcd0a3fc8f0186f42c9100d: Status 404 returned error can't find the container with id 16c08191db3ba8aa73490757d8929d5323fa14c6cdcd0a3fc8f0186f42c9100d
Jul 29 18:26:00.118378 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:00.118347    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk" event={"ID":"e4813cc7-8519-4636-b64f-1832de893f82","Type":"ContainerStarted","Data":"16c08191db3ba8aa73490757d8929d5323fa14c6cdcd0a3fc8f0186f42c9100d"}
Jul 29 18:26:01.125171 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:01.125132    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk" event={"ID":"e4813cc7-8519-4636-b64f-1832de893f82","Type":"ContainerStarted","Data":"4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b"}
Jul 29 18:26:01.142045 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:01.141999    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk" podStartSLOduration=2.141984623 podStartE2EDuration="2.141984623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:25:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:26:01.140784524 +0000 UTC m=+2012.410283320" watchObservedRunningTime="2026-07-29 18:26:01.141984623 +0000 UTC m=+2012.411483403"
Jul 29 18:26:02.130595 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:02.130564    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:26:07.146796 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:07.146761    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:26:11.183118 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:11.183080    2580 generic.go:356] "Generic (PLEG): container finished" podID="912b93b1-1765-4492-8b95-0efd03039d99" containerID="6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb" exitCode=0
Jul 29 18:26:11.183582 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:11.183152    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" event={"ID":"912b93b1-1765-4492-8b95-0efd03039d99","Type":"ContainerDied","Data":"6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb"}
Jul 29 18:26:11.183582 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:11.183547    2580 scope.go:117] "RemoveContainer" containerID="6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb"
Jul 29 18:26:11.184547 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:11.184514    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:26:11.184639 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:11.184600    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:26:11.184705 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:11.184646    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:26:12.192922 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:12.192883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" event={"ID":"912b93b1-1765-4492-8b95-0efd03039d99","Type":"ContainerStarted","Data":"980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848"}
Jul 29 18:26:12.987561 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:12.987519    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:26:12.988800 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:12.988757    2580 patch_prober.go:28] interesting pod/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 29 18:26:12.988943 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:12.988821    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 29 18:26:13.136784 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:13.136758    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:26:13.987932 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:13.987896    2580 patch_prober.go:28] interesting pod/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 29 18:26:13.988393 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:13.988033    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 29 18:26:14.112656 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:14.112631    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:26:14.987865 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:14.987814    2580 patch_prober.go:28] interesting pod/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 29 18:26:14.988079 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:14.987885    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 29 18:26:15.988249 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:15.988217    2580 patch_prober.go:28] interesting pod/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 29 18:26:15.988630 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:15.988268    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 29 18:26:16.991504 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:16.991478    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:26:16.992616 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:16.992600    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:26:29.279242 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:29.279207    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"]
Jul 29 18:26:29.279850 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:29.279445    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk" podUID="e4813cc7-8519-4636-b64f-1832de893f82" containerName="payload-processing" containerID="cri-o://4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b" gracePeriod=30
Jul 29 18:26:29.545204 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:29.545181    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:26:29.607029 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:29.607004    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4813cc7-8519-4636-b64f-1832de893f82-plugins-config-volume\") pod \"e4813cc7-8519-4636-b64f-1832de893f82\" (UID: \"e4813cc7-8519-4636-b64f-1832de893f82\") "
Jul 29 18:26:29.607166 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:29.607095    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgmm5\" (UniqueName: \"kubernetes.io/projected/e4813cc7-8519-4636-b64f-1832de893f82-kube-api-access-lgmm5\") pod \"e4813cc7-8519-4636-b64f-1832de893f82\" (UID: \"e4813cc7-8519-4636-b64f-1832de893f82\") "
Jul 29 18:26:29.607375 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:29.607354    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4813cc7-8519-4636-b64f-1832de893f82-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e4813cc7-8519-4636-b64f-1832de893f82" (UID: "e4813cc7-8519-4636-b64f-1832de893f82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:26:29.607476 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:29.607453    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4813cc7-8519-4636-b64f-1832de893f82-plugins-config-volume\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:26:29.609262 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:29.609241    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4813cc7-8519-4636-b64f-1832de893f82-kube-api-access-lgmm5" (OuterVolumeSpecName: "kube-api-access-lgmm5") pod "e4813cc7-8519-4636-b64f-1832de893f82" (UID: "e4813cc7-8519-4636-b64f-1832de893f82"). InnerVolumeSpecName "kube-api-access-lgmm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:26:29.708019 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:29.707991    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgmm5\" (UniqueName: \"kubernetes.io/projected/e4813cc7-8519-4636-b64f-1832de893f82-kube-api-access-lgmm5\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:26:30.286444 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:30.286414    2580 generic.go:356] "Generic (PLEG): container finished" podID="e4813cc7-8519-4636-b64f-1832de893f82" containerID="4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b" exitCode=0
Jul 29 18:26:30.286919 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:30.286468    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"
Jul 29 18:26:30.286919 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:30.286505    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk" event={"ID":"e4813cc7-8519-4636-b64f-1832de893f82","Type":"ContainerDied","Data":"4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b"}
Jul 29 18:26:30.286919 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:30.286554    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk" event={"ID":"e4813cc7-8519-4636-b64f-1832de893f82","Type":"ContainerDied","Data":"16c08191db3ba8aa73490757d8929d5323fa14c6cdcd0a3fc8f0186f42c9100d"}
Jul 29 18:26:30.286919 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:30.286578    2580 scope.go:117] "RemoveContainer" containerID="4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b"
Jul 29 18:26:30.296004 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:30.295987    2580 scope.go:117] "RemoveContainer" containerID="4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b"
Jul 29 18:26:30.296244 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:26:30.296228    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b\": container with ID starting with 4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b not found: ID does not exist" containerID="4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b"
Jul 29 18:26:30.296314 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:30.296253    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b"} err="failed to get container status \"4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b\": rpc error: code = NotFound desc = could not find container \"4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b\": container with ID starting with 4a64e141cf87f0e9aa05faafb3ee64dd46ed8beaa69bc108dd513e85ffc9382b not found: ID does not exist"
Jul 29 18:26:30.306695 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:30.306671    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"]
Jul 29 18:26:30.315677 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:30.315651    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f861467d-68fbb88d68-5c9jk"]
Jul 29 18:26:30.554878 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:30.554807    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"]
Jul 29 18:26:30.555029 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:30.555009    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy" containerID="cri-o://980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848" gracePeriod=30
Jul 29 18:26:31.272386 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.272344    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4813cc7-8519-4636-b64f-1832de893f82" path="/var/lib/kubelet/pods/e4813cc7-8519-4636-b64f-1832de893f82/volumes"
Jul 29 18:26:31.451755 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.451728    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"]
Jul 29 18:26:31.452175 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.452160    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4813cc7-8519-4636-b64f-1832de893f82" containerName="payload-processing"
Jul 29 18:26:31.452219 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.452193    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4813cc7-8519-4636-b64f-1832de893f82" containerName="payload-processing"
Jul 29 18:26:31.452262 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.452253    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4813cc7-8519-4636-b64f-1832de893f82" containerName="payload-processing"
Jul 29 18:26:31.455167 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.455149    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.457592 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.457572    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-eb3353f1-openshift-default-dockercfg-l4qw7\""
Jul 29 18:26:31.470065 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.470040    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"]
Jul 29 18:26:31.524625 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.524559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-certs\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.524625 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.524592    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-credential-socket\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.524625 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.524612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4721a3b-07e8-4250-b24f-9700fd9addd7-istiod-ca-cert\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.524848 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.524680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-socket\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.524848 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.524737    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-token\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.524848 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.524754    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkfzm\" (UniqueName: \"kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-kube-api-access-kkfzm\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.524848 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.524783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-podinfo\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.525080 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.524892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-data\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.525080 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.524942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-envoy\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626020 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.625989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-socket\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626168 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-token\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626168 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkfzm\" (UniqueName: \"kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-kube-api-access-kkfzm\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626168 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-podinfo\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626327 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-data\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626327 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-envoy\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626435 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-certs\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626435 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-credential-socket\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626435 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4721a3b-07e8-4250-b24f-9700fd9addd7-istiod-ca-cert\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626435 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-socket\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626609 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-data\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626698 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626679    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-certs\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.626847 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.626825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-credential-socket\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.627351 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.627328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4721a3b-07e8-4250-b24f-9700fd9addd7-istiod-ca-cert\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.628609 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.628590    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-envoy\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.628717 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.628696    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-podinfo\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.633935 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.633908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkfzm\" (UniqueName: \"kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-kube-api-access-kkfzm\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.634056 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.633916    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-token\") pod \"e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") " pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.765112 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.765086    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:31.908901 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.908846    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"]
Jul 29 18:26:31.911773 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:26:31.911748    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4721a3b_07e8_4250_b24f_9700fd9addd7.slice/crio-f155dccb6daf443ad2683e48dea14847910ed1635d4f365ac299d96f4515d675 WatchSource:0}: Error finding container f155dccb6daf443ad2683e48dea14847910ed1635d4f365ac299d96f4515d675: Status 404 returned error can't find the container with id f155dccb6daf443ad2683e48dea14847910ed1635d4f365ac299d96f4515d675
Jul 29 18:26:31.914084 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.914051    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:26:31.914154 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.914117    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:26:31.914192 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.914151    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:26:31.993966 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.993919    2580 patch_prober.go:28] interesting pod/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 18:26:31.994076 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:31.993990    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 18:26:32.299884 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:32.299847    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" event={"ID":"d4721a3b-07e8-4250-b24f-9700fd9addd7","Type":"ContainerStarted","Data":"687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91"}
Jul 29 18:26:32.300098 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:32.299896    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" event={"ID":"d4721a3b-07e8-4250-b24f-9700fd9addd7","Type":"ContainerStarted","Data":"f155dccb6daf443ad2683e48dea14847910ed1635d4f365ac299d96f4515d675"}
Jul 29 18:26:32.317052 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:32.316986    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" podStartSLOduration=1.316950221 podStartE2EDuration="1.316950221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:26:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:26:32.316358847 +0000 UTC m=+2043.585857631" watchObservedRunningTime="2026-07-29 18:26:32.316950221 +0000 UTC m=+2043.586448996"
Jul 29 18:26:32.765821 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:32.765733    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:32.770189 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:32.770164    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:33.201681 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:33.201643    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:26:33.305285 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:33.305249    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:33.306265 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:33.306246    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:35.814211 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.814146    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:26:35.970474 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970448    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/912b93b1-1765-4492-8b95-0efd03039d99-istiod-ca-cert\") pod \"912b93b1-1765-4492-8b95-0efd03039d99\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") "
Jul 29 18:26:35.970668 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970525    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-certs\") pod \"912b93b1-1765-4492-8b95-0efd03039d99\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") "
Jul 29 18:26:35.970668 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970547    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/912b93b1-1765-4492-8b95-0efd03039d99-istio-podinfo\") pod \"912b93b1-1765-4492-8b95-0efd03039d99\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") "
Jul 29 18:26:35.970668 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970585    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-socket\") pod \"912b93b1-1765-4492-8b95-0efd03039d99\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") "
Jul 29 18:26:35.970668 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970617    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-credential-socket\") pod \"912b93b1-1765-4492-8b95-0efd03039d99\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") "
Jul 29 18:26:35.970668 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970647    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-data\") pod \"912b93b1-1765-4492-8b95-0efd03039d99\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") "
Jul 29 18:26:35.970942 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970671    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpmm7\" (UniqueName: \"kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-kube-api-access-cpmm7\") pod \"912b93b1-1765-4492-8b95-0efd03039d99\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") "
Jul 29 18:26:35.970942 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970697    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-istio-token\") pod \"912b93b1-1765-4492-8b95-0efd03039d99\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") "
Jul 29 18:26:35.970942 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970727    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-envoy\") pod \"912b93b1-1765-4492-8b95-0efd03039d99\" (UID: \"912b93b1-1765-4492-8b95-0efd03039d99\") "
Jul 29 18:26:35.970942 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970823    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "912b93b1-1765-4492-8b95-0efd03039d99" (UID: "912b93b1-1765-4492-8b95-0efd03039d99"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:26:35.970942 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970925    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "912b93b1-1765-4492-8b95-0efd03039d99" (UID: "912b93b1-1765-4492-8b95-0efd03039d99"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:26:35.971240 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.970989    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/912b93b1-1765-4492-8b95-0efd03039d99-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "912b93b1-1765-4492-8b95-0efd03039d99" (UID: "912b93b1-1765-4492-8b95-0efd03039d99"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:26:35.971240 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.971141    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:26:35.971240 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.971165    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:26:35.971240 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.971178    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/912b93b1-1765-4492-8b95-0efd03039d99-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:26:35.971417 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.971354    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "912b93b1-1765-4492-8b95-0efd03039d99" (UID: "912b93b1-1765-4492-8b95-0efd03039d99"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:26:35.971682 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.971635    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-data" (OuterVolumeSpecName: "istio-data") pod "912b93b1-1765-4492-8b95-0efd03039d99" (UID: "912b93b1-1765-4492-8b95-0efd03039d99"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:26:35.973049 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.973013    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/912b93b1-1765-4492-8b95-0efd03039d99-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "912b93b1-1765-4492-8b95-0efd03039d99" (UID: "912b93b1-1765-4492-8b95-0efd03039d99"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:26:35.973152 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.973080    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-istio-token" (OuterVolumeSpecName: "istio-token") pod "912b93b1-1765-4492-8b95-0efd03039d99" (UID: "912b93b1-1765-4492-8b95-0efd03039d99"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:26:35.973471 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.973454    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "912b93b1-1765-4492-8b95-0efd03039d99" (UID: "912b93b1-1765-4492-8b95-0efd03039d99"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:26:35.973536 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:35.973464    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-kube-api-access-cpmm7" (OuterVolumeSpecName: "kube-api-access-cpmm7") pod "912b93b1-1765-4492-8b95-0efd03039d99" (UID: "912b93b1-1765-4492-8b95-0efd03039d99"). InnerVolumeSpecName "kube-api-access-cpmm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:26:36.072307 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.072254    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:26:36.072307 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.072273    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:26:36.072307 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.072283    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpmm7\" (UniqueName: \"kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-kube-api-access-cpmm7\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:26:36.072307 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.072292    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/912b93b1-1765-4492-8b95-0efd03039d99-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:26:36.072307 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.072301    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/912b93b1-1765-4492-8b95-0efd03039d99-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:26:36.072307 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.072308    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/912b93b1-1765-4492-8b95-0efd03039d99-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:26:36.320836 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.320804    2580 generic.go:356] "Generic (PLEG): container finished" podID="912b93b1-1765-4492-8b95-0efd03039d99" containerID="980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848" exitCode=0
Jul 29 18:26:36.321012 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.320879    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"
Jul 29 18:26:36.321012 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.320880    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" event={"ID":"912b93b1-1765-4492-8b95-0efd03039d99","Type":"ContainerDied","Data":"980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848"}
Jul 29 18:26:36.321012 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.320942    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx" event={"ID":"912b93b1-1765-4492-8b95-0efd03039d99","Type":"ContainerDied","Data":"a325f0ec14bb280c34ed9e4c18f73a6280416ddb737939d87e29e9c35068a520"}
Jul 29 18:26:36.321012 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.320982    2580 scope.go:117] "RemoveContainer" containerID="980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848"
Jul 29 18:26:36.331657 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.331638    2580 scope.go:117] "RemoveContainer" containerID="6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb"
Jul 29 18:26:36.340123 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.340103    2580 scope.go:117] "RemoveContainer" containerID="980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848"
Jul 29 18:26:36.340525 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:26:36.340499    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848\": container with ID starting with 980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848 not found: ID does not exist" containerID="980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848"
Jul 29 18:26:36.340677 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.340562    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848"} err="failed to get container status \"980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848\": rpc error: code = NotFound desc = could not find container \"980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848\": container with ID starting with 980914c38c212cd8f68389fede90ec9855c54fc4a2eee71af296d6a3ef16f848 not found: ID does not exist"
Jul 29 18:26:36.340677 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.340588    2580 scope.go:117] "RemoveContainer" containerID="6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb"
Jul 29 18:26:36.341171 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:26:36.341131    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb\": container with ID starting with 6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb not found: ID does not exist" containerID="6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb"
Jul 29 18:26:36.341319 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.341168    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb"} err="failed to get container status \"6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb\": rpc error: code = NotFound desc = could not find container \"6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb\": container with ID starting with 6d0a3cbb148952a0b52ad4caea1931ae05405ddba6be91dcd7a342433e0364eb not found: ID does not exist"
Jul 29 18:26:36.343177 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.343154    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"]
Jul 29 18:26:36.350657 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:36.350636    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f861467d-openshift-default-5959694b6d-xcnxx"]
Jul 29 18:26:37.279093 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:37.279051    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="912b93b1-1765-4492-8b95-0efd03039d99" path="/var/lib/kubelet/pods/912b93b1-1765-4492-8b95-0efd03039d99/volumes"
Jul 29 18:26:44.105514 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:44.105478    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:26:48.307204 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:48.307165    2580 patch_prober.go:28] interesting pod/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 29 18:26:48.307713 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:48.307239    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 29 18:26:48.379780 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:48.379750    2580 generic.go:356] "Generic (PLEG): container finished" podID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerID="687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91" exitCode=0
Jul 29 18:26:48.379979 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:48.379832    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" event={"ID":"d4721a3b-07e8-4250-b24f-9700fd9addd7","Type":"ContainerDied","Data":"687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91"}
Jul 29 18:26:48.380246 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:48.380232    2580 scope.go:117] "RemoveContainer" containerID="687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91"
Jul 29 18:26:48.381261 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:48.381234    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:26:48.381343 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:48.381317    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:26:48.381385 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:48.381352    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:26:49.390395 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:49.390353    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" event={"ID":"d4721a3b-07e8-4250-b24f-9700fd9addd7","Type":"ContainerStarted","Data":"0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e"}
Jul 29 18:26:49.778043 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:49.771108    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:49.778043 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:49.771494    2580 patch_prober.go:28] interesting pod/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 29 18:26:49.778043 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:49.771540    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 29 18:26:50.766168 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:50.766079    2580 patch_prober.go:28] interesting pod/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 29 18:26:50.766733 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:50.766700    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 29 18:26:51.765669 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:51.765632    2580 patch_prober.go:28] interesting pod/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 29 18:26:51.765988 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:51.765933    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 29 18:26:52.765620 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:52.765580    2580 patch_prober.go:28] interesting pod/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 29 18:26:52.766141 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:52.765642    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 29 18:26:53.769583 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:53.769556    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:53.769991 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:53.769916    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:26:53.770646 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:26:53.770628    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:27:04.626671 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:04.626640    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"]
Jul 29 18:27:04.627450 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:04.626865    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy" containerID="cri-o://0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e" gracePeriod=30
Jul 29 18:27:08.294843 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:08.294804    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:27:08.771880 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:08.771805    2580 patch_prober.go:28] interesting pod/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 18:27:08.771880 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:08.771857    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 18:27:09.883114 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.883085    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:27:09.981723 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.981644    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-podinfo\") pod \"d4721a3b-07e8-4250-b24f-9700fd9addd7\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") "
Jul 29 18:27:09.981723 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.981677    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkfzm\" (UniqueName: \"kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-kube-api-access-kkfzm\") pod \"d4721a3b-07e8-4250-b24f-9700fd9addd7\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") "
Jul 29 18:27:09.981723 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.981705    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-certs\") pod \"d4721a3b-07e8-4250-b24f-9700fd9addd7\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") "
Jul 29 18:27:09.982061 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.981827    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4721a3b-07e8-4250-b24f-9700fd9addd7-istiod-ca-cert\") pod \"d4721a3b-07e8-4250-b24f-9700fd9addd7\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") "
Jul 29 18:27:09.982061 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.981930    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-socket\") pod \"d4721a3b-07e8-4250-b24f-9700fd9addd7\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") "
Jul 29 18:27:09.982061 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.982000    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-token\") pod \"d4721a3b-07e8-4250-b24f-9700fd9addd7\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") "
Jul 29 18:27:09.982061 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.982024    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-envoy\") pod \"d4721a3b-07e8-4250-b24f-9700fd9addd7\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") "
Jul 29 18:27:09.982061 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.982045    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4721a3b-07e8-4250-b24f-9700fd9addd7" (UID: "d4721a3b-07e8-4250-b24f-9700fd9addd7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:27:09.982061 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.982063    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-credential-socket\") pod \"d4721a3b-07e8-4250-b24f-9700fd9addd7\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") "
Jul 29 18:27:09.982350 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.982095    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-data\") pod \"d4721a3b-07e8-4250-b24f-9700fd9addd7\" (UID: \"d4721a3b-07e8-4250-b24f-9700fd9addd7\") "
Jul 29 18:27:09.982350 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.982183    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4721a3b-07e8-4250-b24f-9700fd9addd7" (UID: "d4721a3b-07e8-4250-b24f-9700fd9addd7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:27:09.982350 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.982211    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4721a3b-07e8-4250-b24f-9700fd9addd7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4721a3b-07e8-4250-b24f-9700fd9addd7" (UID: "d4721a3b-07e8-4250-b24f-9700fd9addd7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:27:09.982555 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.982536    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:27:09.982609 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.982562    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4721a3b-07e8-4250-b24f-9700fd9addd7-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:27:09.982609 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.982576    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:27:09.982716 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.982691    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4721a3b-07e8-4250-b24f-9700fd9addd7" (UID: "d4721a3b-07e8-4250-b24f-9700fd9addd7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:27:09.983254 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.983228    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4721a3b-07e8-4250-b24f-9700fd9addd7" (UID: "d4721a3b-07e8-4250-b24f-9700fd9addd7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:27:09.984101 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.984081    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-kube-api-access-kkfzm" (OuterVolumeSpecName: "kube-api-access-kkfzm") pod "d4721a3b-07e8-4250-b24f-9700fd9addd7" (UID: "d4721a3b-07e8-4250-b24f-9700fd9addd7"). InnerVolumeSpecName "kube-api-access-kkfzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:27:09.984229 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.984212    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4721a3b-07e8-4250-b24f-9700fd9addd7" (UID: "d4721a3b-07e8-4250-b24f-9700fd9addd7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:27:09.984303 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.984283    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4721a3b-07e8-4250-b24f-9700fd9addd7" (UID: "d4721a3b-07e8-4250-b24f-9700fd9addd7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:27:09.984542 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:09.984517    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4721a3b-07e8-4250-b24f-9700fd9addd7" (UID: "d4721a3b-07e8-4250-b24f-9700fd9addd7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:27:10.084084 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.084052    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:27:10.084084 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.084077    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:27:10.084084 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.084087    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:27:10.084084 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.084095    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:27:10.084342 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.084104    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4721a3b-07e8-4250-b24f-9700fd9addd7-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:27:10.084342 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.084111    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkfzm\" (UniqueName: \"kubernetes.io/projected/d4721a3b-07e8-4250-b24f-9700fd9addd7-kube-api-access-kkfzm\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:27:10.498550 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.498511    2580 generic.go:356] "Generic (PLEG): container finished" podID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerID="0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e" exitCode=0
Jul 29 18:27:10.498720 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.498558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" event={"ID":"d4721a3b-07e8-4250-b24f-9700fd9addd7","Type":"ContainerDied","Data":"0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e"}
Jul 29 18:27:10.498720 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.498566    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"
Jul 29 18:27:10.498720 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.498625    2580 scope.go:117] "RemoveContainer" containerID="0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e"
Jul 29 18:27:10.498720 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.498614    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5" event={"ID":"d4721a3b-07e8-4250-b24f-9700fd9addd7","Type":"ContainerDied","Data":"f155dccb6daf443ad2683e48dea14847910ed1635d4f365ac299d96f4515d675"}
Jul 29 18:27:10.507802 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.507787    2580 scope.go:117] "RemoveContainer" containerID="687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91"
Jul 29 18:27:10.515671 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.515657    2580 scope.go:117] "RemoveContainer" containerID="0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e"
Jul 29 18:27:10.515903 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:27:10.515881    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e\": container with ID starting with 0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e not found: ID does not exist" containerID="0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e"
Jul 29 18:27:10.516009 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.515914    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e"} err="failed to get container status \"0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e\": rpc error: code = NotFound desc = could not find container \"0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e\": container with ID starting with 0a1a2ea92cc5271b172191321f124ec642189c540c77d4305af612c9d304947e not found: ID does not exist"
Jul 29 18:27:10.516009 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.515937    2580 scope.go:117] "RemoveContainer" containerID="687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91"
Jul 29 18:27:10.516456 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:27:10.516433    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91\": container with ID starting with 687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91 not found: ID does not exist" containerID="687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91"
Jul 29 18:27:10.516538 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.516462    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91"} err="failed to get container status \"687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91\": rpc error: code = NotFound desc = could not find container \"687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91\": container with ID starting with 687fd2e6ae1c16ea5eabd37e0c0ad9992945335b95130fd86a8f95aa265b2a91 not found: ID does not exist"
Jul 29 18:27:10.519000 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.518980    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"]
Jul 29 18:27:10.522301 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:10.522282    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-eb3353f1-openshift-default-5f7d476868-kmcm5"]
Jul 29 18:27:11.272207 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:11.272177    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" path="/var/lib/kubelet/pods/d4721a3b-07e8-4250-b24f-9700fd9addd7/volumes"
Jul 29 18:27:31.099081 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:27:31.099045    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:28:59.611136 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:28:59.611059    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:31:04.503159 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:04.503128    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:31:07.184935 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.184892    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"]
Jul 29 18:31:07.185717 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.185698    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy"
Jul 29 18:31:07.185762 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.185722    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy"
Jul 29 18:31:07.185762 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.185737    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy"
Jul 29 18:31:07.185762 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.185745    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy"
Jul 29 18:31:07.185855 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.185764    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy"
Jul 29 18:31:07.185855 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.185775    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy"
Jul 29 18:31:07.185923 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.185889    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy"
Jul 29 18:31:07.185923 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.185900    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="912b93b1-1765-4492-8b95-0efd03039d99" containerName="istio-proxy"
Jul 29 18:31:07.185923 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.185911    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy"
Jul 29 18:31:07.189529 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.189511    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.192369 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.192317    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 18:31:07.192530 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.192322    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-96b81602-openshift-default-dockercfg-6qmzr\""
Jul 29 18:31:07.192530 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.192446    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 18:31:07.192643 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.192632    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 18:31:07.201565 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.201544    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"]
Jul 29 18:31:07.331216 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.331187    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-token\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.331216 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.331219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-envoy\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.331396 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.331248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67f29229-875a-4cbd-b6be-aaec6a5931e8-istiod-ca-cert\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.331396 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.331276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-podinfo\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.331396 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.331304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9xx4\" (UniqueName: \"kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-kube-api-access-g9xx4\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.331396 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.331352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-socket\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.331396 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.331389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-certs\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.331555 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.331426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-data\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.331555 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.331448    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-credential-socket\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.432706 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.432678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-data\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.432864 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.432713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-credential-socket\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.432864 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.432742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-token\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.432864 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.432769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-envoy\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.432864 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.432804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67f29229-875a-4cbd-b6be-aaec6a5931e8-istiod-ca-cert\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.432864 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.432832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-podinfo\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.432864 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.432862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9xx4\" (UniqueName: \"kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-kube-api-access-g9xx4\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.433228 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.432916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-socket\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.433228 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.433132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-certs\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.433228 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.433179    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-credential-socket\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.433392 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.433297    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-data\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.433542 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.433514    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-socket\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.433797 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.433777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-certs\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.434008 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.433992    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67f29229-875a-4cbd-b6be-aaec6a5931e8-istiod-ca-cert\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.435418 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.435367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-podinfo\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.435418 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.435401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-envoy\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.440840 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.440812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-token\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.441153 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.441134    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9xx4\" (UniqueName: \"kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-kube-api-access-g9xx4\") pod \"e2e-mt-96b81602-openshift-default-7cb5564856-twfvp\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") " pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.500454 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.500432    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:07.664320 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.664295    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"]
Jul 29 18:31:07.666565 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:31:07.666532    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f29229_875a_4cbd_b6be_aaec6a5931e8.slice/crio-7f6d02b6c23499edc81f3762c82c852b9d88ef24d2df181518ae5c8edc9e0cc1 WatchSource:0}: Error finding container 7f6d02b6c23499edc81f3762c82c852b9d88ef24d2df181518ae5c8edc9e0cc1: Status 404 returned error can't find the container with id 7f6d02b6c23499edc81f3762c82c852b9d88ef24d2df181518ae5c8edc9e0cc1
Jul 29 18:31:07.668861 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.668779    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:31:07.668996 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.668930    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:31:07.669060 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.669039    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:31:07.680177 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:07.680157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" event={"ID":"67f29229-875a-4cbd-b6be-aaec6a5931e8","Type":"ContainerStarted","Data":"7f6d02b6c23499edc81f3762c82c852b9d88ef24d2df181518ae5c8edc9e0cc1"}
Jul 29 18:31:08.686797 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:08.686763    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" event={"ID":"67f29229-875a-4cbd-b6be-aaec6a5931e8","Type":"ContainerStarted","Data":"b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df"}
Jul 29 18:31:08.708163 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:08.708110    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" podStartSLOduration=1.70809503 podStartE2EDuration="1.70809503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:31:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:31:08.70643583 +0000 UTC m=+2319.975934623" watchObservedRunningTime="2026-07-29 18:31:08.70809503 +0000 UTC m=+2319.977593812"
Jul 29 18:31:09.500834 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:09.500799    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:09.505694 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:09.505665    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:09.692195 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:09.692170    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:09.693087 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:09.693070    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:11.559842 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.559809    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"]
Jul 29 18:31:11.560266 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.560252    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy"
Jul 29 18:31:11.560310 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.560270    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy"
Jul 29 18:31:11.560374 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.560364    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4721a3b-07e8-4250-b24f-9700fd9addd7" containerName="istio-proxy"
Jul 29 18:31:11.563470 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.563452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:11.565657 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.565633    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-96b81602\""
Jul 29 18:31:11.565657 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.565640    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-96b81602-dockercfg-zhf96\""
Jul 29 18:31:11.571401 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.571375    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"]
Jul 29 18:31:11.670157 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.670125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/523e5629-c2e8-4c1b-946d-bbb294823cf8-plugins-config-volume\") pod \"payload-processing-e2e-mt-96b81602-8697fbd87d-b886j\" (UID: \"523e5629-c2e8-4c1b-946d-bbb294823cf8\") " pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:11.670306 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.670247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s2xc\" (UniqueName: \"kubernetes.io/projected/523e5629-c2e8-4c1b-946d-bbb294823cf8-kube-api-access-4s2xc\") pod \"payload-processing-e2e-mt-96b81602-8697fbd87d-b886j\" (UID: \"523e5629-c2e8-4c1b-946d-bbb294823cf8\") " pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:11.770686 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.770657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s2xc\" (UniqueName: \"kubernetes.io/projected/523e5629-c2e8-4c1b-946d-bbb294823cf8-kube-api-access-4s2xc\") pod \"payload-processing-e2e-mt-96b81602-8697fbd87d-b886j\" (UID: \"523e5629-c2e8-4c1b-946d-bbb294823cf8\") " pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:11.770830 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.770715    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/523e5629-c2e8-4c1b-946d-bbb294823cf8-plugins-config-volume\") pod \"payload-processing-e2e-mt-96b81602-8697fbd87d-b886j\" (UID: \"523e5629-c2e8-4c1b-946d-bbb294823cf8\") " pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:11.771347 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.771331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/523e5629-c2e8-4c1b-946d-bbb294823cf8-plugins-config-volume\") pod \"payload-processing-e2e-mt-96b81602-8697fbd87d-b886j\" (UID: \"523e5629-c2e8-4c1b-946d-bbb294823cf8\") " pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:11.778519 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.778493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s2xc\" (UniqueName: \"kubernetes.io/projected/523e5629-c2e8-4c1b-946d-bbb294823cf8-kube-api-access-4s2xc\") pod \"payload-processing-e2e-mt-96b81602-8697fbd87d-b886j\" (UID: \"523e5629-c2e8-4c1b-946d-bbb294823cf8\") " pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:11.876246 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:11.876190    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:12.020049 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:12.020020    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"]
Jul 29 18:31:12.021289 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:31:12.021263    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod523e5629_c2e8_4c1b_946d_bbb294823cf8.slice/crio-9b53b70e32877f0619584b9468b24faf960c59f757ee25e04f1bb2e5bab36f8c WatchSource:0}: Error finding container 9b53b70e32877f0619584b9468b24faf960c59f757ee25e04f1bb2e5bab36f8c: Status 404 returned error can't find the container with id 9b53b70e32877f0619584b9468b24faf960c59f757ee25e04f1bb2e5bab36f8c
Jul 29 18:31:12.708711 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:12.708681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j" event={"ID":"523e5629-c2e8-4c1b-946d-bbb294823cf8","Type":"ContainerStarted","Data":"8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d"}
Jul 29 18:31:12.709127 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:12.708735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j" event={"ID":"523e5629-c2e8-4c1b-946d-bbb294823cf8","Type":"ContainerStarted","Data":"9b53b70e32877f0619584b9468b24faf960c59f757ee25e04f1bb2e5bab36f8c"}
Jul 29 18:31:12.709127 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:12.708816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:12.724770 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:12.724720    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j" podStartSLOduration=1.724704826 podStartE2EDuration="1.724704826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:31:12.723968389 +0000 UTC m=+2323.993467169" watchObservedRunningTime="2026-07-29 18:31:12.724704826 +0000 UTC m=+2323.994203607"
Jul 29 18:31:18.797656 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:18.797623    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:31:23.716455 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:23.716364    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:25.733113 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:25.733074    2580 patch_prober.go:28] interesting pod/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 18:31:25.733486 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:25.733136    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 18:31:27.794115 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:27.794086    2580 generic.go:356] "Generic (PLEG): container finished" podID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerID="b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df" exitCode=0
Jul 29 18:31:27.794562 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:27.794161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" event={"ID":"67f29229-875a-4cbd-b6be-aaec6a5931e8","Type":"ContainerDied","Data":"b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df"}
Jul 29 18:31:27.794562 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:27.794521    2580 scope.go:117] "RemoveContainer" containerID="b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df"
Jul 29 18:31:27.795491 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:27.795460    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:31:27.795586 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:27.795528    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:31:27.795586 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:27.795562    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:31:28.801363 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:28.801326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" event={"ID":"67f29229-875a-4cbd-b6be-aaec6a5931e8","Type":"ContainerStarted","Data":"9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1"}
Jul 29 18:31:29.500795 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:29.500759    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:29.501024 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:29.500946    2580 patch_prober.go:28] interesting pod/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 18:31:29.501116 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:29.501033    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 18:31:30.501687 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:30.501651    2580 patch_prober.go:28] interesting pod/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 18:31:30.502084 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:30.501734    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 18:31:31.501409 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:31.501368    2580 patch_prober.go:28] interesting pod/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 18:31:31.501591 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:31.501451    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 18:31:32.501192 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:32.501151    2580 patch_prober.go:28] interesting pod/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 29 18:31:32.501584 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:32.501216    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 29 18:31:33.504076 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:33.504040    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:33.504555 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:33.504429    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:33.505124 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:33.505104    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:31:53.481206 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.481169    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"]
Jul 29 18:31:53.481828 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.481447    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j" podUID="523e5629-c2e8-4c1b-946d-bbb294823cf8" containerName="payload-processing" containerID="cri-o://8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d" gracePeriod=30
Jul 29 18:31:53.736436 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.736368    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:53.846672 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.846639    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4s2xc\" (UniqueName: \"kubernetes.io/projected/523e5629-c2e8-4c1b-946d-bbb294823cf8-kube-api-access-4s2xc\") pod \"523e5629-c2e8-4c1b-946d-bbb294823cf8\" (UID: \"523e5629-c2e8-4c1b-946d-bbb294823cf8\") "
Jul 29 18:31:53.846864 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.846699    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/523e5629-c2e8-4c1b-946d-bbb294823cf8-plugins-config-volume\") pod \"523e5629-c2e8-4c1b-946d-bbb294823cf8\" (UID: \"523e5629-c2e8-4c1b-946d-bbb294823cf8\") "
Jul 29 18:31:53.847157 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.847131    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/523e5629-c2e8-4c1b-946d-bbb294823cf8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "523e5629-c2e8-4c1b-946d-bbb294823cf8" (UID: "523e5629-c2e8-4c1b-946d-bbb294823cf8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:31:53.848938 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.848915    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/523e5629-c2e8-4c1b-946d-bbb294823cf8-kube-api-access-4s2xc" (OuterVolumeSpecName: "kube-api-access-4s2xc") pod "523e5629-c2e8-4c1b-946d-bbb294823cf8" (UID: "523e5629-c2e8-4c1b-946d-bbb294823cf8"). InnerVolumeSpecName "kube-api-access-4s2xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:31:53.927405 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.927368    2580 generic.go:356] "Generic (PLEG): container finished" podID="523e5629-c2e8-4c1b-946d-bbb294823cf8" containerID="8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d" exitCode=0
Jul 29 18:31:53.927588 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.927412    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"
Jul 29 18:31:53.927588 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.927458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j" event={"ID":"523e5629-c2e8-4c1b-946d-bbb294823cf8","Type":"ContainerDied","Data":"8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d"}
Jul 29 18:31:53.927588 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.927512    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j" event={"ID":"523e5629-c2e8-4c1b-946d-bbb294823cf8","Type":"ContainerDied","Data":"9b53b70e32877f0619584b9468b24faf960c59f757ee25e04f1bb2e5bab36f8c"}
Jul 29 18:31:53.927588 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.927536    2580 scope.go:117] "RemoveContainer" containerID="8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d"
Jul 29 18:31:53.936378 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.936361    2580 scope.go:117] "RemoveContainer" containerID="8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d"
Jul 29 18:31:53.936639 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:31:53.936621    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d\": container with ID starting with 8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d not found: ID does not exist" containerID="8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d"
Jul 29 18:31:53.936688 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.936647    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d"} err="failed to get container status \"8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d\": rpc error: code = NotFound desc = could not find container \"8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d\": container with ID starting with 8619061737bd92b54848ac5bb6b99876229ffc1b2b8e6a0a0d93434f9feeba6d not found: ID does not exist"
Jul 29 18:31:53.948177 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.948157    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s2xc\" (UniqueName: \"kubernetes.io/projected/523e5629-c2e8-4c1b-946d-bbb294823cf8-kube-api-access-4s2xc\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:31:53.948177 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.948178    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/523e5629-c2e8-4c1b-946d-bbb294823cf8-plugins-config-volume\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:31:53.948933 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.948914    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"]
Jul 29 18:31:53.953193 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:53.953175    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j"]
Jul 29 18:31:54.715149 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:54.715105    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-96b81602-8697fbd87d-b886j" podUID="523e5629-c2e8-4c1b-946d-bbb294823cf8" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.43:9004: i/o timeout"
Jul 29 18:31:55.272819 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:55.272786    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="523e5629-c2e8-4c1b-946d-bbb294823cf8" path="/var/lib/kubelet/pods/523e5629-c2e8-4c1b-946d-bbb294823cf8/volumes"
Jul 29 18:31:56.296467 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:31:56.296423    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:32:00.278433 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:00.278397    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"]
Jul 29 18:32:00.278916 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:00.278685    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy" containerID="cri-o://9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1" gracePeriod=30
Jul 29 18:32:03.507314 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:03.507276    2580 patch_prober.go:28] interesting pod/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 18:32:03.507711 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:03.507336    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 18:32:05.535329 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.535301    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:32:05.646821 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.646733    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67f29229-875a-4cbd-b6be-aaec6a5931e8-istiod-ca-cert\") pod \"67f29229-875a-4cbd-b6be-aaec6a5931e8\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") "
Jul 29 18:32:05.646821 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.646766    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-data\") pod \"67f29229-875a-4cbd-b6be-aaec6a5931e8\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") "
Jul 29 18:32:05.646821 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.646799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-socket\") pod \"67f29229-875a-4cbd-b6be-aaec6a5931e8\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") "
Jul 29 18:32:05.646821 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.646816    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-envoy\") pod \"67f29229-875a-4cbd-b6be-aaec6a5931e8\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") "
Jul 29 18:32:05.647202 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.646855    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-token\") pod \"67f29229-875a-4cbd-b6be-aaec6a5931e8\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") "
Jul 29 18:32:05.647202 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.646871    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9xx4\" (UniqueName: \"kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-kube-api-access-g9xx4\") pod \"67f29229-875a-4cbd-b6be-aaec6a5931e8\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") "
Jul 29 18:32:05.647202 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.646918    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-podinfo\") pod \"67f29229-875a-4cbd-b6be-aaec6a5931e8\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") "
Jul 29 18:32:05.647202 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.647036    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-credential-socket\") pod \"67f29229-875a-4cbd-b6be-aaec6a5931e8\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") "
Jul 29 18:32:05.647202 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.647074    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-certs\") pod \"67f29229-875a-4cbd-b6be-aaec6a5931e8\" (UID: \"67f29229-875a-4cbd-b6be-aaec6a5931e8\") "
Jul 29 18:32:05.647202 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.647116    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67f29229-875a-4cbd-b6be-aaec6a5931e8" (UID: "67f29229-875a-4cbd-b6be-aaec6a5931e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:32:05.647202 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.647178    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67f29229-875a-4cbd-b6be-aaec6a5931e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67f29229-875a-4cbd-b6be-aaec6a5931e8" (UID: "67f29229-875a-4cbd-b6be-aaec6a5931e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:32:05.647553 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.647398    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67f29229-875a-4cbd-b6be-aaec6a5931e8" (UID: "67f29229-875a-4cbd-b6be-aaec6a5931e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:32:05.647625 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.647552    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:32:05.647625 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.647559    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67f29229-875a-4cbd-b6be-aaec6a5931e8" (UID: "67f29229-875a-4cbd-b6be-aaec6a5931e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:32:05.647625 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.647572    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67f29229-875a-4cbd-b6be-aaec6a5931e8-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:32:05.647625 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.647625    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:32:05.648011 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.647770    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "67f29229-875a-4cbd-b6be-aaec6a5931e8" (UID: "67f29229-875a-4cbd-b6be-aaec6a5931e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:32:05.649359 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.649331    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "67f29229-875a-4cbd-b6be-aaec6a5931e8" (UID: "67f29229-875a-4cbd-b6be-aaec6a5931e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:32:05.649477 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.649363    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67f29229-875a-4cbd-b6be-aaec6a5931e8" (UID: "67f29229-875a-4cbd-b6be-aaec6a5931e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:32:05.649728 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.649708    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67f29229-875a-4cbd-b6be-aaec6a5931e8" (UID: "67f29229-875a-4cbd-b6be-aaec6a5931e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:32:05.649779 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.649724    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-kube-api-access-g9xx4" (OuterVolumeSpecName: "kube-api-access-g9xx4") pod "67f29229-875a-4cbd-b6be-aaec6a5931e8" (UID: "67f29229-875a-4cbd-b6be-aaec6a5931e8"). InnerVolumeSpecName "kube-api-access-g9xx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:32:05.749139 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.749106    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:32:05.749139 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.749134    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9xx4\" (UniqueName: \"kubernetes.io/projected/67f29229-875a-4cbd-b6be-aaec6a5931e8-kube-api-access-g9xx4\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:32:05.749139 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.749146    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:32:05.749369 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.749154    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:32:05.749369 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.749163    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:32:05.749369 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.749171    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67f29229-875a-4cbd-b6be-aaec6a5931e8-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:32:05.989688 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.989600    2580 generic.go:356] "Generic (PLEG): container finished" podID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerID="9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1" exitCode=0
Jul 29 18:32:05.989688 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.989647    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"
Jul 29 18:32:05.989919 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.989732    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" event={"ID":"67f29229-875a-4cbd-b6be-aaec6a5931e8","Type":"ContainerDied","Data":"9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1"}
Jul 29 18:32:05.989919 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.989792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp" event={"ID":"67f29229-875a-4cbd-b6be-aaec6a5931e8","Type":"ContainerDied","Data":"7f6d02b6c23499edc81f3762c82c852b9d88ef24d2df181518ae5c8edc9e0cc1"}
Jul 29 18:32:05.989919 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:05.989821    2580 scope.go:117] "RemoveContainer" containerID="9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1"
Jul 29 18:32:06.001751 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:06.001733    2580 scope.go:117] "RemoveContainer" containerID="b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df"
Jul 29 18:32:06.010529 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:06.010494    2580 scope.go:117] "RemoveContainer" containerID="9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1"
Jul 29 18:32:06.011325 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:32:06.011279    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1\": container with ID starting with 9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1 not found: ID does not exist" containerID="9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1"
Jul 29 18:32:06.011428 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:06.011321    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1"} err="failed to get container status \"9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1\": rpc error: code = NotFound desc = could not find container \"9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1\": container with ID starting with 9f5e566845f5fe62eebeeec3ed4f5a470018101c20463487316c68314fe1c3e1 not found: ID does not exist"
Jul 29 18:32:06.011428 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:06.011343    2580 scope.go:117] "RemoveContainer" containerID="b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df"
Jul 29 18:32:06.011643 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:32:06.011616    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df\": container with ID starting with b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df not found: ID does not exist" containerID="b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df"
Jul 29 18:32:06.011756 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:06.011649    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df"} err="failed to get container status \"b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df\": rpc error: code = NotFound desc = could not find container \"b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df\": container with ID starting with b128019d47c5580a7f231ed30fe9495fb1c3b27a049e29c09af5776f7644b6df not found: ID does not exist"
Jul 29 18:32:06.013103 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:06.013084    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"]
Jul 29 18:32:06.016493 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:06.016474    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-96b81602-openshift-default-7cb5564856-twfvp"]
Jul 29 18:32:07.273451 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:07.273415    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" path="/var/lib/kubelet/pods/67f29229-875a-4cbd-b6be-aaec6a5931e8/volumes"
Jul 29 18:32:36.101408 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:32:36.101372    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:34:16.292798 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.292766    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"]
Jul 29 18:34:16.293420 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.293386    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="523e5629-c2e8-4c1b-946d-bbb294823cf8" containerName="payload-processing"
Jul 29 18:34:16.293420 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.293406    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="523e5629-c2e8-4c1b-946d-bbb294823cf8" containerName="payload-processing"
Jul 29 18:34:16.293545 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.293423    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy"
Jul 29 18:34:16.293545 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.293432    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy"
Jul 29 18:34:16.293545 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.293536    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy"
Jul 29 18:34:16.293737 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.293548    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="523e5629-c2e8-4c1b-946d-bbb294823cf8" containerName="payload-processing"
Jul 29 18:34:16.293737 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.293560    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy"
Jul 29 18:34:16.296513 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.296489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.298700 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.298678    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8be27fe2-openshift-default-dockercfg-nzsvp\""
Jul 29 18:34:16.298838 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.298770    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 18:34:16.298912 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.298850    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 18:34:16.299027 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.299016    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 18:34:16.308292 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.308269    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"]
Jul 29 18:34:16.364833 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.364798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-istio-token\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.365088 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.364842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-certs\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.365088 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.364865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-credential-socket\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.365088 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.364941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/073eb412-5e3a-43db-968f-94b553c18d02-istiod-ca-cert\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.365088 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.365001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/073eb412-5e3a-43db-968f-94b553c18d02-istio-podinfo\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.365088 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.365017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-data\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.365088 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.365088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-envoy\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.365427 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.365108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-socket\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.365427 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.365129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsr6p\" (UniqueName: \"kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-kube-api-access-bsr6p\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.466201 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.466170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-data\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.466412 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.466272    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-envoy\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.466412 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.466306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-socket\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.466412 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.466338    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsr6p\" (UniqueName: \"kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-kube-api-access-bsr6p\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.466412 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.466384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-istio-token\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.466653 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.466418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-certs\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.466653 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.466451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-credential-socket\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.466653 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.466495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/073eb412-5e3a-43db-968f-94b553c18d02-istiod-ca-cert\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.466814 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.466685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/073eb412-5e3a-43db-968f-94b553c18d02-istio-podinfo\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.466814 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.466622    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-data\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.466926 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.466881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-credential-socket\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.467024 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.467001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-certs\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.467024 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.467014    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-socket\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.467521 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.467497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/073eb412-5e3a-43db-968f-94b553c18d02-istiod-ca-cert\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.468777 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.468757    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-envoy\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.469328 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.469304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/073eb412-5e3a-43db-968f-94b553c18d02-istio-podinfo\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.474833 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.474809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsr6p\" (UniqueName: \"kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-kube-api-access-bsr6p\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.474935 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.474920    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-istio-token\") pod \"e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") " pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.608219 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.608123    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:16.763944 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.763912    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"]
Jul 29 18:34:16.765167 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:34:16.765135    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073eb412_5e3a_43db_968f_94b553c18d02.slice/crio-5e99eb9c9ee1bd1b7f1603d2486612cd6956121cca01a2716d00137ccffa1cbe WatchSource:0}: Error finding container 5e99eb9c9ee1bd1b7f1603d2486612cd6956121cca01a2716d00137ccffa1cbe: Status 404 returned error can't find the container with id 5e99eb9c9ee1bd1b7f1603d2486612cd6956121cca01a2716d00137ccffa1cbe
Jul 29 18:34:16.767435 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.767403    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:34:16.767542 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.767468    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:34:16.767542 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:16.767498    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:34:17.640356 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:17.640319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" event={"ID":"073eb412-5e3a-43db-968f-94b553c18d02","Type":"ContainerStarted","Data":"3ddc2281cd3b613971bc073a82d670e27edd9f6be0d5582eba605c92e4de34d4"}
Jul 29 18:34:17.640774 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:17.640377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" event={"ID":"073eb412-5e3a-43db-968f-94b553c18d02","Type":"ContainerStarted","Data":"5e99eb9c9ee1bd1b7f1603d2486612cd6956121cca01a2716d00137ccffa1cbe"}
Jul 29 18:34:17.663110 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:17.663047    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" podStartSLOduration=1.663026248 podStartE2EDuration="1.663026248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:34:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:17.662083588 +0000 UTC m=+2508.931582373" watchObservedRunningTime="2026-07-29 18:34:17.663026248 +0000 UTC m=+2508.932525032"
Jul 29 18:34:18.609045 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:18.608949    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:18.613481 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:18.613456    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:18.645642 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:18.645611    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:18.646701 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:18.646675    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:20.015740 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.015696    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:34:20.639490 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.639458    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"]
Jul 29 18:34:20.640020 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.640001    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy"
Jul 29 18:34:20.640020 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.640020    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="67f29229-875a-4cbd-b6be-aaec6a5931e8" containerName="istio-proxy"
Jul 29 18:34:20.643370 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.643356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:34:20.645340 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.645318    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8be27fe2\""
Jul 29 18:34:20.646563 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.646537    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8be27fe2-dockercfg-jpngc\""
Jul 29 18:34:20.652896 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.652875    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"]
Jul 29 18:34:20.710948 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.710914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65wqc\" (UniqueName: \"kubernetes.io/projected/5fc45d75-a638-496d-b4e3-60103ada1662-kube-api-access-65wqc\") pod \"payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9\" (UID: \"5fc45d75-a638-496d-b4e3-60103ada1662\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:34:20.711171 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.711122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc45d75-a638-496d-b4e3-60103ada1662-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9\" (UID: \"5fc45d75-a638-496d-b4e3-60103ada1662\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:34:20.811821 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.811782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc45d75-a638-496d-b4e3-60103ada1662-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9\" (UID: \"5fc45d75-a638-496d-b4e3-60103ada1662\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:34:20.812060 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.811865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65wqc\" (UniqueName: \"kubernetes.io/projected/5fc45d75-a638-496d-b4e3-60103ada1662-kube-api-access-65wqc\") pod \"payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9\" (UID: \"5fc45d75-a638-496d-b4e3-60103ada1662\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:34:20.812517 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.812497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc45d75-a638-496d-b4e3-60103ada1662-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9\" (UID: \"5fc45d75-a638-496d-b4e3-60103ada1662\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:34:20.819689 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.819671    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65wqc\" (UniqueName: \"kubernetes.io/projected/5fc45d75-a638-496d-b4e3-60103ada1662-kube-api-access-65wqc\") pod \"payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9\" (UID: \"5fc45d75-a638-496d-b4e3-60103ada1662\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:34:20.956228 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:20.956133    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:34:21.098040 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:21.097927    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"]
Jul 29 18:34:21.100276 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:34:21.100238    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc45d75_a638_496d_b4e3_60103ada1662.slice/crio-53f5f584419f5e21ad048d74beac708cb0eb9e2079c592522a2fb34233bc21b2 WatchSource:0}: Error finding container 53f5f584419f5e21ad048d74beac708cb0eb9e2079c592522a2fb34233bc21b2: Status 404 returned error can't find the container with id 53f5f584419f5e21ad048d74beac708cb0eb9e2079c592522a2fb34233bc21b2
Jul 29 18:34:21.662770 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:21.662733    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9" event={"ID":"5fc45d75-a638-496d-b4e3-60103ada1662","Type":"ContainerStarted","Data":"99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721"}
Jul 29 18:34:21.662994 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:21.662793    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9" event={"ID":"5fc45d75-a638-496d-b4e3-60103ada1662","Type":"ContainerStarted","Data":"53f5f584419f5e21ad048d74beac708cb0eb9e2079c592522a2fb34233bc21b2"}
Jul 29 18:34:21.662994 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:21.662853    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:34:21.682765 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:21.682706    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9" podStartSLOduration=1.682687526 podStartE2EDuration="1.682687526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:34:21.682212125 +0000 UTC m=+2512.951710931" watchObservedRunningTime="2026-07-29 18:34:21.682687526 +0000 UTC m=+2512.952186308"
Jul 29 18:34:32.671186 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:32.671146    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:34:32.918194 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:32.918159    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:34:34.687147 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:34.687113    2580 patch_prober.go:28] interesting pod/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 18:34:34.687561 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:34.687185    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 18:34:36.739472 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:36.739382    2580 generic.go:356] "Generic (PLEG): container finished" podID="073eb412-5e3a-43db-968f-94b553c18d02" containerID="3ddc2281cd3b613971bc073a82d670e27edd9f6be0d5582eba605c92e4de34d4" exitCode=0
Jul 29 18:34:36.739472 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:36.739418    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" event={"ID":"073eb412-5e3a-43db-968f-94b553c18d02","Type":"ContainerDied","Data":"3ddc2281cd3b613971bc073a82d670e27edd9f6be0d5582eba605c92e4de34d4"}
Jul 29 18:34:36.739864 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:36.739764    2580 scope.go:117] "RemoveContainer" containerID="3ddc2281cd3b613971bc073a82d670e27edd9f6be0d5582eba605c92e4de34d4"
Jul 29 18:34:36.740684 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:36.740657    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:34:36.740739 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:36.740723    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:34:36.740775 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:36.740753    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:34:37.747698 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:37.747663    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" event={"ID":"073eb412-5e3a-43db-968f-94b553c18d02","Type":"ContainerStarted","Data":"3d81c227263d8dff77cea78aaf49e57e1b0f5816fc3fa1a461b0a484bb56530f"}
Jul 29 18:34:38.609289 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:38.609254    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:38.609480 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:38.609458    2580 patch_prober.go:28] interesting pod/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 18:34:38.609534 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:38.609518    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 18:34:39.609105 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:39.609072    2580 patch_prober.go:28] interesting pod/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 18:34:39.609481 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:39.609135    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 18:34:40.608914 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:40.608877    2580 patch_prober.go:28] interesting pod/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 18:34:40.609112 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:40.608930    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 18:34:41.609135 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:41.609092    2580 patch_prober.go:28] interesting pod/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 18:34:41.609621 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:41.609165    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 18:34:42.612223 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:42.612186    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:42.612668 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:42.612462    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:42.613138 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:42.613121    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:34:59.604580 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:34:59.604543    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:35:13.800488 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:13.800452    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:35:38.297592 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:38.297557    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:35:40.082168 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:40.082123    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"]
Jul 29 18:35:40.082649 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:40.082330    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9" podUID="5fc45d75-a638-496d-b4e3-60103ada1662" containerName="payload-pre-processing" containerID="cri-o://99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721" gracePeriod=30
Jul 29 18:35:40.343848 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:40.343789    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:35:40.514368 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:40.514338    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc45d75-a638-496d-b4e3-60103ada1662-plugins-config-volume\") pod \"5fc45d75-a638-496d-b4e3-60103ada1662\" (UID: \"5fc45d75-a638-496d-b4e3-60103ada1662\") "
Jul 29 18:35:40.514542 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:40.514390    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65wqc\" (UniqueName: \"kubernetes.io/projected/5fc45d75-a638-496d-b4e3-60103ada1662-kube-api-access-65wqc\") pod \"5fc45d75-a638-496d-b4e3-60103ada1662\" (UID: \"5fc45d75-a638-496d-b4e3-60103ada1662\") "
Jul 29 18:35:40.514709 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:40.514686    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fc45d75-a638-496d-b4e3-60103ada1662-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5fc45d75-a638-496d-b4e3-60103ada1662" (UID: "5fc45d75-a638-496d-b4e3-60103ada1662"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:35:40.516548 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:40.516524    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fc45d75-a638-496d-b4e3-60103ada1662-kube-api-access-65wqc" (OuterVolumeSpecName: "kube-api-access-65wqc") pod "5fc45d75-a638-496d-b4e3-60103ada1662" (UID: "5fc45d75-a638-496d-b4e3-60103ada1662"). InnerVolumeSpecName "kube-api-access-65wqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:35:40.615418 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:40.615353    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc45d75-a638-496d-b4e3-60103ada1662-plugins-config-volume\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:35:40.615418 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:40.615379    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65wqc\" (UniqueName: \"kubernetes.io/projected/5fc45d75-a638-496d-b4e3-60103ada1662-kube-api-access-65wqc\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:35:41.061971 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:41.061930    2580 generic.go:356] "Generic (PLEG): container finished" podID="5fc45d75-a638-496d-b4e3-60103ada1662" containerID="99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721" exitCode=0
Jul 29 18:35:41.062134 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:41.061976    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9" event={"ID":"5fc45d75-a638-496d-b4e3-60103ada1662","Type":"ContainerDied","Data":"99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721"}
Jul 29 18:35:41.062134 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:41.062045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9" event={"ID":"5fc45d75-a638-496d-b4e3-60103ada1662","Type":"ContainerDied","Data":"53f5f584419f5e21ad048d74beac708cb0eb9e2079c592522a2fb34233bc21b2"}
Jul 29 18:35:41.062134 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:41.061994    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"
Jul 29 18:35:41.062134 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:41.062077    2580 scope.go:117] "RemoveContainer" containerID="99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721"
Jul 29 18:35:41.071397 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:41.071382    2580 scope.go:117] "RemoveContainer" containerID="99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721"
Jul 29 18:35:41.071655 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:35:41.071636    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721\": container with ID starting with 99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721 not found: ID does not exist" containerID="99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721"
Jul 29 18:35:41.071729 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:41.071662    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721"} err="failed to get container status \"99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721\": rpc error: code = NotFound desc = could not find container \"99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721\": container with ID starting with 99ad54fd47d53b385a269134125f4be2ba8536a018c3bb3071a71a7fb225a721 not found: ID does not exist"
Jul 29 18:35:41.083288 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:41.083254    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"]
Jul 29 18:35:41.088362 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:41.088337    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8be27fe2-78d98d9d4-bbxd9"]
Jul 29 18:35:41.272397 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:41.272358    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fc45d75-a638-496d-b4e3-60103ada1662" path="/var/lib/kubelet/pods/5fc45d75-a638-496d-b4e3-60103ada1662/volumes"
Jul 29 18:35:48.496412 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:35:48.496370    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:36:46.222135 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:46.222104    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"]
Jul 29 18:36:46.222658 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:46.222310    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy" containerID="cri-o://3d81c227263d8dff77cea78aaf49e57e1b0f5816fc3fa1a461b0a484bb56530f" gracePeriod=30
Jul 29 18:36:48.702353 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:48.702315    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:36:51.417360 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.417330    2580 generic.go:356] "Generic (PLEG): container finished" podID="073eb412-5e3a-43db-968f-94b553c18d02" containerID="3d81c227263d8dff77cea78aaf49e57e1b0f5816fc3fa1a461b0a484bb56530f" exitCode=0
Jul 29 18:36:51.417727 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.417413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" event={"ID":"073eb412-5e3a-43db-968f-94b553c18d02","Type":"ContainerDied","Data":"3d81c227263d8dff77cea78aaf49e57e1b0f5816fc3fa1a461b0a484bb56530f"}
Jul 29 18:36:51.417727 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.417477    2580 scope.go:117] "RemoveContainer" containerID="3ddc2281cd3b613971bc073a82d670e27edd9f6be0d5582eba605c92e4de34d4"
Jul 29 18:36:51.475827 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.475808    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:36:51.498855 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.498789    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-socket\") pod \"073eb412-5e3a-43db-968f-94b553c18d02\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") "
Jul 29 18:36:51.498855 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.498830    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-credential-socket\") pod \"073eb412-5e3a-43db-968f-94b553c18d02\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") "
Jul 29 18:36:51.499070 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.498858    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-certs\") pod \"073eb412-5e3a-43db-968f-94b553c18d02\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") "
Jul 29 18:36:51.499070 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.498896    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/073eb412-5e3a-43db-968f-94b553c18d02-istiod-ca-cert\") pod \"073eb412-5e3a-43db-968f-94b553c18d02\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") "
Jul 29 18:36:51.499070 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.498936    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-data\") pod \"073eb412-5e3a-43db-968f-94b553c18d02\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") "
Jul 29 18:36:51.499070 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.498972    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-istio-token\") pod \"073eb412-5e3a-43db-968f-94b553c18d02\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") "
Jul 29 18:36:51.499070 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.499001    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/073eb412-5e3a-43db-968f-94b553c18d02-istio-podinfo\") pod \"073eb412-5e3a-43db-968f-94b553c18d02\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") "
Jul 29 18:36:51.499070 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.499022    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-envoy\") pod \"073eb412-5e3a-43db-968f-94b553c18d02\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") "
Jul 29 18:36:51.499070 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.499050    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsr6p\" (UniqueName: \"kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-kube-api-access-bsr6p\") pod \"073eb412-5e3a-43db-968f-94b553c18d02\" (UID: \"073eb412-5e3a-43db-968f-94b553c18d02\") "
Jul 29 18:36:51.499795 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.499668    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "073eb412-5e3a-43db-968f-94b553c18d02" (UID: "073eb412-5e3a-43db-968f-94b553c18d02"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:36:51.501074 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.500525    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "073eb412-5e3a-43db-968f-94b553c18d02" (UID: "073eb412-5e3a-43db-968f-94b553c18d02"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:36:51.501074 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.500721    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "073eb412-5e3a-43db-968f-94b553c18d02" (UID: "073eb412-5e3a-43db-968f-94b553c18d02"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:36:51.501074 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.500981    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-data" (OuterVolumeSpecName: "istio-data") pod "073eb412-5e3a-43db-968f-94b553c18d02" (UID: "073eb412-5e3a-43db-968f-94b553c18d02"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:36:51.501074 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.501029    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/073eb412-5e3a-43db-968f-94b553c18d02-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "073eb412-5e3a-43db-968f-94b553c18d02" (UID: "073eb412-5e3a-43db-968f-94b553c18d02"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:36:51.502065 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.502022    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-kube-api-access-bsr6p" (OuterVolumeSpecName: "kube-api-access-bsr6p") pod "073eb412-5e3a-43db-968f-94b553c18d02" (UID: "073eb412-5e3a-43db-968f-94b553c18d02"). InnerVolumeSpecName "kube-api-access-bsr6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:36:51.502384 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.502358    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/073eb412-5e3a-43db-968f-94b553c18d02-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "073eb412-5e3a-43db-968f-94b553c18d02" (UID: "073eb412-5e3a-43db-968f-94b553c18d02"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:36:51.503254 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.503227    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-istio-token" (OuterVolumeSpecName: "istio-token") pod "073eb412-5e3a-43db-968f-94b553c18d02" (UID: "073eb412-5e3a-43db-968f-94b553c18d02"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:36:51.503497 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.503475    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "073eb412-5e3a-43db-968f-94b553c18d02" (UID: "073eb412-5e3a-43db-968f-94b553c18d02"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:36:51.600439 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.600412    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:36:51.600439 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.600434    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:36:51.600580 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.600444    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:36:51.600580 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.600453    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/073eb412-5e3a-43db-968f-94b553c18d02-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:36:51.600580 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.600461    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:36:51.600580 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.600469    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:36:51.600580 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.600476    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/073eb412-5e3a-43db-968f-94b553c18d02-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:36:51.600580 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.600483    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/073eb412-5e3a-43db-968f-94b553c18d02-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:36:51.600580 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:51.600493    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsr6p\" (UniqueName: \"kubernetes.io/projected/073eb412-5e3a-43db-968f-94b553c18d02-kube-api-access-bsr6p\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:36:52.423215 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:52.423186    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"
Jul 29 18:36:52.423720 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:52.423227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6" event={"ID":"073eb412-5e3a-43db-968f-94b553c18d02","Type":"ContainerDied","Data":"5e99eb9c9ee1bd1b7f1603d2486612cd6956121cca01a2716d00137ccffa1cbe"}
Jul 29 18:36:52.423720 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:52.423285    2580 scope.go:117] "RemoveContainer" containerID="3d81c227263d8dff77cea78aaf49e57e1b0f5816fc3fa1a461b0a484bb56530f"
Jul 29 18:36:52.444837 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:52.444813    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"]
Jul 29 18:36:52.450335 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:52.450314    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8be27fe2-openshift-default-55fc7c8d68-jv5q6"]
Jul 29 18:36:53.272819 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:36:53.272785    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="073eb412-5e3a-43db-968f-94b553c18d02" path="/var/lib/kubelet/pods/073eb412-5e3a-43db-968f-94b553c18d02/volumes"
Jul 29 18:37:02.302263 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:37:02.302226    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:37:23.502737 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:37:23.502644    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:37:32.447334 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:37:32.447300    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:39:56.059326 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.059292    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"]
Jul 29 18:39:56.061715 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.059732    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy"
Jul 29 18:39:56.061715 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.059743    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy"
Jul 29 18:39:56.061715 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.059769    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fc45d75-a638-496d-b4e3-60103ada1662" containerName="payload-pre-processing"
Jul 29 18:39:56.061715 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.059774    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fc45d75-a638-496d-b4e3-60103ada1662" containerName="payload-pre-processing"
Jul 29 18:39:56.061715 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.059835    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy"
Jul 29 18:39:56.061715 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.059843    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fc45d75-a638-496d-b4e3-60103ada1662" containerName="payload-pre-processing"
Jul 29 18:39:56.061715 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.059851    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy"
Jul 29 18:39:56.062707 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.062693    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.071024 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.071001    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 18:39:56.071171 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.071157    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-54bd5a-openshift-default-dockercfg-v7b8b\""
Jul 29 18:39:56.071426 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.071414    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 18:39:56.071749 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.071734    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 18:39:56.136103 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.136080    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"]
Jul 29 18:39:56.169136 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.169105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-socket\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.169265 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.169143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbp54\" (UniqueName: \"kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-kube-api-access-jbp54\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.169265 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.169175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-credential-socket\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.169265 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.169197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-data\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.169383 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.169259    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-podinfo\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.169383 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.169309    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-certs\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.169383 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.169332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-envoy\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.169383 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.169363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-token\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.169533 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.169409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istiod-ca-cert\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270228 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-socket\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270363 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbp54\" (UniqueName: \"kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-kube-api-access-jbp54\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270363 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-credential-socket\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270363 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-data\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270531 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-podinfo\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270531 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-certs\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270531 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-envoy\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270531 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-token\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270731 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istiod-ca-cert\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270731 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-socket\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270731 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-credential-socket\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270893 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-data\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.270985 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.270903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-certs\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.271243 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.271222    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istiod-ca-cert\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.272798 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.272775    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-envoy\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.272947 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.272929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-podinfo\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.284942 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.284925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-token\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.285355 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.285335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbp54\" (UniqueName: \"kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-kube-api-access-jbp54\") pod \"e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") " pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.373306 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.373242    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:56.529833 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.529806    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"]
Jul 29 18:39:56.532336 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:39:56.532311    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2827c94_1702_40e2_bb05_5fbef2c0d72b.slice/crio-f209a10167c13ba92d95934dc2d177f1a33971d13ecaadb16f585c3f66191452 WatchSource:0}: Error finding container f209a10167c13ba92d95934dc2d177f1a33971d13ecaadb16f585c3f66191452: Status 404 returned error can't find the container with id f209a10167c13ba92d95934dc2d177f1a33971d13ecaadb16f585c3f66191452
Jul 29 18:39:56.534416 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.534382    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:39:56.534513 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.534449    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:39:56.534513 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:56.534500    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:39:57.322777 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:57.322738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr" event={"ID":"d2827c94-1702-40e2-bb05-5fbef2c0d72b","Type":"ContainerStarted","Data":"d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05"}
Jul 29 18:39:57.323207 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:57.322792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr" event={"ID":"d2827c94-1702-40e2-bb05-5fbef2c0d72b","Type":"ContainerStarted","Data":"f209a10167c13ba92d95934dc2d177f1a33971d13ecaadb16f585c3f66191452"}
Jul 29 18:39:57.340895 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:57.340838    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr" podStartSLOduration=1.340820122 podStartE2EDuration="1.340820122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:39:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:39:57.340092559 +0000 UTC m=+2848.609591363" watchObservedRunningTime="2026-07-29 18:39:57.340820122 +0000 UTC m=+2848.610318909"
Jul 29 18:39:57.373414 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:57.373391    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:57.378538 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:57.378516    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:58.328239 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:58.328213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:39:58.329279 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:39:58.329262    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:40:00.200888 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.200850    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"]
Jul 29 18:40:00.201413 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.201397    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy"
Jul 29 18:40:00.201413 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.201414    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="073eb412-5e3a-43db-968f-94b553c18d02" containerName="istio-proxy"
Jul 29 18:40:00.204492 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.204478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:40:00.206828 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.206804    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-54bd5a\""
Jul 29 18:40:00.206984 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.206903    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-54bd5a-dockercfg-99zmz\""
Jul 29 18:40:00.215678 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.215652    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"]
Jul 29 18:40:00.309423 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.309392    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf2sz\" (UniqueName: \"kubernetes.io/projected/52a9cb5d-ae14-47d1-b567-7878744f7dd6-kube-api-access-gf2sz\") pod \"payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5\" (UID: \"52a9cb5d-ae14-47d1-b567-7878744f7dd6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:40:00.309594 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.309437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52a9cb5d-ae14-47d1-b567-7878744f7dd6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5\" (UID: \"52a9cb5d-ae14-47d1-b567-7878744f7dd6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:40:00.410631 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.410600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf2sz\" (UniqueName: \"kubernetes.io/projected/52a9cb5d-ae14-47d1-b567-7878744f7dd6-kube-api-access-gf2sz\") pod \"payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5\" (UID: \"52a9cb5d-ae14-47d1-b567-7878744f7dd6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:40:00.410814 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.410653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52a9cb5d-ae14-47d1-b567-7878744f7dd6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5\" (UID: \"52a9cb5d-ae14-47d1-b567-7878744f7dd6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:40:00.411599 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.411574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52a9cb5d-ae14-47d1-b567-7878744f7dd6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5\" (UID: \"52a9cb5d-ae14-47d1-b567-7878744f7dd6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:40:00.425785 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.425748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf2sz\" (UniqueName: \"kubernetes.io/projected/52a9cb5d-ae14-47d1-b567-7878744f7dd6-kube-api-access-gf2sz\") pod \"payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5\" (UID: \"52a9cb5d-ae14-47d1-b567-7878744f7dd6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:40:00.516851 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.516767    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:40:00.655980 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:00.655944    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"]
Jul 29 18:40:00.657670 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:40:00.657643    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a9cb5d_ae14_47d1_b567_7878744f7dd6.slice/crio-b0010c6ec7ce1ca045e3bc6651d1eae75dd5597c852c7d8ec3a4e26404004762 WatchSource:0}: Error finding container b0010c6ec7ce1ca045e3bc6651d1eae75dd5597c852c7d8ec3a4e26404004762: Status 404 returned error can't find the container with id b0010c6ec7ce1ca045e3bc6651d1eae75dd5597c852c7d8ec3a4e26404004762
Jul 29 18:40:01.297468 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:01.297429    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:40:01.346553 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:01.346514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5" event={"ID":"52a9cb5d-ae14-47d1-b567-7878744f7dd6","Type":"ContainerStarted","Data":"0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336"}
Jul 29 18:40:01.346740 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:01.346567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5" event={"ID":"52a9cb5d-ae14-47d1-b567-7878744f7dd6","Type":"ContainerStarted","Data":"b0010c6ec7ce1ca045e3bc6651d1eae75dd5597c852c7d8ec3a4e26404004762"}
Jul 29 18:40:01.346740 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:01.346656    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:40:01.361346 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:01.361283    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5" podStartSLOduration=1.361267794 podStartE2EDuration="1.361267794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:40:01.360463049 +0000 UTC m=+2852.629961853" watchObservedRunningTime="2026-07-29 18:40:01.361267794 +0000 UTC m=+2852.630766577"
Jul 29 18:40:09.400031 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:09.399941    2580 generic.go:356] "Generic (PLEG): container finished" podID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerID="d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05" exitCode=0
Jul 29 18:40:09.400031 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:09.400017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr" event={"ID":"d2827c94-1702-40e2-bb05-5fbef2c0d72b","Type":"ContainerDied","Data":"d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05"}
Jul 29 18:40:09.400449 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:09.400405    2580 scope.go:117] "RemoveContainer" containerID="d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05"
Jul 29 18:40:09.401520 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:09.401492    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:40:09.401586 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:09.401560    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:40:09.401622 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:09.401595    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:40:10.407318 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:10.407280    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr" event={"ID":"d2827c94-1702-40e2-bb05-5fbef2c0d72b","Type":"ContainerStarted","Data":"f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182"}
Jul 29 18:40:11.374515 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:11.374478    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:40:11.376128 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:11.376100    2580 patch_prober.go:28] interesting pod/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 29 18:40:11.376335 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:11.376310    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr" podUID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 29 18:40:12.354841 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:12.354816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:40:12.374491 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:12.374438    2580 patch_prober.go:28] interesting pod/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 29 18:40:12.374736 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:12.374713    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr" podUID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 29 18:40:13.328615 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:13.328580    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:40:13.374424 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:13.374385    2580 patch_prober.go:28] interesting pod/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Jul 29 18:40:13.374891 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:13.374462    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr" podUID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Jul 29 18:40:14.378080 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:14.378051    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:40:14.379044 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:14.379023    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:40:24.846745 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.846659    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"]
Jul 29 18:40:24.850107 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.850088    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:24.852208 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.852190    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-084953-openshift-default-dockercfg-h5692\""
Jul 29 18:40:24.860996 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.860975    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"]
Jul 29 18:40:24.946507 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.946473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-credential-socket\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:24.946650 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.946516    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istiod-ca-cert\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:24.946650 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.946569    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-token\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:24.946650 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.946600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-podinfo\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:24.946650 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.946646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgg8v\" (UniqueName: \"kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-kube-api-access-hgg8v\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:24.946872 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.946696    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-data\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:24.946872 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.946733    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-socket\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:24.946872 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.946760    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-certs\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:24.946872 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:24.946802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-envoy\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.047358 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.047323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-credential-socket\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.047501 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.047368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istiod-ca-cert\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.047501 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.047403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-token\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.047501 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.047433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-podinfo\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.047501 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.047477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgg8v\" (UniqueName: \"kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-kube-api-access-hgg8v\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.047716 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.047518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-data\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.047716 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.047556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-socket\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.047716 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.047583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-certs\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.047716 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.047621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-envoy\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.047922 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.047717    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-credential-socket\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.048063 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.048039    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-data\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.048288 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.048191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istiod-ca-cert\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.048392 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.048208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-socket\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.048392 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.048297    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-certs\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.049950 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.049931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-envoy\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.050249 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.050233    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-podinfo\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.055396 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.055377    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-token\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.055565 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.055548    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgg8v\" (UniqueName: \"kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-kube-api-access-hgg8v\") pod \"e2e-inf-b-084953-openshift-default-6587c9778-cx6b6\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") " pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.161094 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.161038    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:25.297650 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.297339    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"]
Jul 29 18:40:25.299877 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:40:25.299852    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79373f51_e9bd_4a13_bc3b_ae6364aa3733.slice/crio-6a491c94d7a5c9844ddfff1fe3012fe4fb3c2d60e5e7a1a02b282bdd9aee75c8 WatchSource:0}: Error finding container 6a491c94d7a5c9844ddfff1fe3012fe4fb3c2d60e5e7a1a02b282bdd9aee75c8: Status 404 returned error can't find the container with id 6a491c94d7a5c9844ddfff1fe3012fe4fb3c2d60e5e7a1a02b282bdd9aee75c8
Jul 29 18:40:25.302014 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.301984    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:40:25.302093 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.302047    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:40:25.302093 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.302076    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:40:25.484142 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.484053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" event={"ID":"79373f51-e9bd-4a13-bc3b-ae6364aa3733","Type":"ContainerStarted","Data":"459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa"}
Jul 29 18:40:25.484142 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.484141    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" event={"ID":"79373f51-e9bd-4a13-bc3b-ae6364aa3733","Type":"ContainerStarted","Data":"6a491c94d7a5c9844ddfff1fe3012fe4fb3c2d60e5e7a1a02b282bdd9aee75c8"}
Jul 29 18:40:25.505730 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:25.505668    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" podStartSLOduration=1.505653589 podStartE2EDuration="1.505653589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:40:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:40:25.502776523 +0000 UTC m=+2876.772275308" watchObservedRunningTime="2026-07-29 18:40:25.505653589 +0000 UTC m=+2876.775152371"
Jul 29 18:40:26.161219 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:26.161184    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:26.166018 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:26.165994    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:26.489449 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:26.489374    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:26.490245 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:26.490224    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:34.193270 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:34.193235    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:40:42.381892 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:42.381352    2580 patch_prober.go:28] interesting pod/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": read tcp 10.134.0.2:40140->10.134.0.48:15021: read: connection reset by peer" start-of-body=
Jul 29 18:40:42.381892 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:42.381431    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": read tcp 10.134.0.2:40140->10.134.0.48:15021: read: connection reset by peer"
Jul 29 18:40:42.572393 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:42.572330    2580 generic.go:356] "Generic (PLEG): container finished" podID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerID="459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa" exitCode=0
Jul 29 18:40:42.572517 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:42.572407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" event={"ID":"79373f51-e9bd-4a13-bc3b-ae6364aa3733","Type":"ContainerDied","Data":"459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa"}
Jul 29 18:40:42.572806 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:42.572793    2580 scope.go:117] "RemoveContainer" containerID="459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa"
Jul 29 18:40:42.573849 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:42.573821    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:40:42.573917 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:42.573907    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:40:42.573991 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:42.573945    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:40:43.583432 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:43.583385    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" event={"ID":"79373f51-e9bd-4a13-bc3b-ae6364aa3733","Type":"ContainerStarted","Data":"43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa"}
Jul 29 18:40:44.161467 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:44.161431    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:44.161668 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:44.161643    2580 patch_prober.go:28] interesting pod/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 29 18:40:44.161753 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:44.161700    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 29 18:40:45.161443 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:45.161408    2580 patch_prober.go:28] interesting pod/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 29 18:40:45.161833 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:45.161463    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 29 18:40:46.162073 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:46.162032    2580 patch_prober.go:28] interesting pod/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 29 18:40:46.162476 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:46.162099    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 29 18:40:47.162231 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:47.162192    2580 patch_prober.go:28] interesting pod/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Jul 29 18:40:47.162734 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:47.162247    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Jul 29 18:40:48.165356 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:48.165324    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:48.165784 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:48.165717    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:40:48.166349 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:40:48.166329    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:41:06.995883 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:41:06.995848    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:41:18.191204 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:41:18.191171    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:41:39.494438 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:41:39.494398    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:41:52.486919 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:41:52.486837    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:42:22.494734 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:22.494700    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:42:28.651811 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:28.651773    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"]
Jul 29 18:42:28.652431 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:28.652117    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5" podUID="52a9cb5d-ae14-47d1-b567-7878744f7dd6" containerName="payload-processing" containerID="cri-o://0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336" gracePeriod=30
Jul 29 18:42:28.908804 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:28.908748    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:42:29.076829 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.076798    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52a9cb5d-ae14-47d1-b567-7878744f7dd6-plugins-config-volume\") pod \"52a9cb5d-ae14-47d1-b567-7878744f7dd6\" (UID: \"52a9cb5d-ae14-47d1-b567-7878744f7dd6\") "
Jul 29 18:42:29.077022 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.076860    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf2sz\" (UniqueName: \"kubernetes.io/projected/52a9cb5d-ae14-47d1-b567-7878744f7dd6-kube-api-access-gf2sz\") pod \"52a9cb5d-ae14-47d1-b567-7878744f7dd6\" (UID: \"52a9cb5d-ae14-47d1-b567-7878744f7dd6\") "
Jul 29 18:42:29.077197 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.077174    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52a9cb5d-ae14-47d1-b567-7878744f7dd6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "52a9cb5d-ae14-47d1-b567-7878744f7dd6" (UID: "52a9cb5d-ae14-47d1-b567-7878744f7dd6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:42:29.079019 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.078994    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a9cb5d-ae14-47d1-b567-7878744f7dd6-kube-api-access-gf2sz" (OuterVolumeSpecName: "kube-api-access-gf2sz") pod "52a9cb5d-ae14-47d1-b567-7878744f7dd6" (UID: "52a9cb5d-ae14-47d1-b567-7878744f7dd6"). InnerVolumeSpecName "kube-api-access-gf2sz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:29.131382 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.131359    2580 generic.go:356] "Generic (PLEG): container finished" podID="52a9cb5d-ae14-47d1-b567-7878744f7dd6" containerID="0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336" exitCode=0
Jul 29 18:42:29.131518 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.131398    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"
Jul 29 18:42:29.131518 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.131416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5" event={"ID":"52a9cb5d-ae14-47d1-b567-7878744f7dd6","Type":"ContainerDied","Data":"0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336"}
Jul 29 18:42:29.131518 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.131480    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5" event={"ID":"52a9cb5d-ae14-47d1-b567-7878744f7dd6","Type":"ContainerDied","Data":"b0010c6ec7ce1ca045e3bc6651d1eae75dd5597c852c7d8ec3a4e26404004762"}
Jul 29 18:42:29.131518 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.131514    2580 scope.go:117] "RemoveContainer" containerID="0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336"
Jul 29 18:42:29.140852 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.140834    2580 scope.go:117] "RemoveContainer" containerID="0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336"
Jul 29 18:42:29.141138 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:42:29.141119    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336\": container with ID starting with 0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336 not found: ID does not exist" containerID="0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336"
Jul 29 18:42:29.141199 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.141146    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336"} err="failed to get container status \"0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336\": rpc error: code = NotFound desc = could not find container \"0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336\": container with ID starting with 0f8e794c1361d7591780b711fe165520456ec8f6997ea6fef8e06578026b0336 not found: ID does not exist"
Jul 29 18:42:29.150920 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.150900    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"]
Jul 29 18:42:29.154397 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.154378    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-54bd5a-544f77bb8-5mwk5"]
Jul 29 18:42:29.178109 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.178059    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52a9cb5d-ae14-47d1-b567-7878744f7dd6-plugins-config-volume\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:29.178109 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.178078    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf2sz\" (UniqueName: \"kubernetes.io/projected/52a9cb5d-ae14-47d1-b567-7878744f7dd6-kube-api-access-gf2sz\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:29.273254 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:29.273229    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52a9cb5d-ae14-47d1-b567-7878744f7dd6" path="/var/lib/kubelet/pods/52a9cb5d-ae14-47d1-b567-7878744f7dd6/volumes"
Jul 29 18:42:35.260753 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:35.260725    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"]
Jul 29 18:42:35.261254 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:35.260906    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr" podUID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerName="istio-proxy" containerID="cri-o://f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182" gracePeriod=30
Jul 29 18:42:37.687496 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:37.687461    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:42:40.519913 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.519889    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:42:40.686568 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.686532    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-envoy\") pod \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") "
Jul 29 18:42:40.686771 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.686590    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-podinfo\") pod \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") "
Jul 29 18:42:40.686771 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.686654    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-socket\") pod \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") "
Jul 29 18:42:40.686771 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.686677    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-certs\") pod \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") "
Jul 29 18:42:40.686945 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.686850    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbp54\" (UniqueName: \"kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-kube-api-access-jbp54\") pod \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") "
Jul 29 18:42:40.686945 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.686885    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d2827c94-1702-40e2-bb05-5fbef2c0d72b" (UID: "d2827c94-1702-40e2-bb05-5fbef2c0d72b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:40.686945 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.686906    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-credential-socket\") pod \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") "
Jul 29 18:42:40.687135 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.686950    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istiod-ca-cert\") pod \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") "
Jul 29 18:42:40.687135 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.687038    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-data\") pod \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") "
Jul 29 18:42:40.687135 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.687088    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-token\") pod \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\" (UID: \"d2827c94-1702-40e2-bb05-5fbef2c0d72b\") "
Jul 29 18:42:40.687135 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.687036    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d2827c94-1702-40e2-bb05-5fbef2c0d72b" (UID: "d2827c94-1702-40e2-bb05-5fbef2c0d72b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:40.687360 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.687112    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d2827c94-1702-40e2-bb05-5fbef2c0d72b" (UID: "d2827c94-1702-40e2-bb05-5fbef2c0d72b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:40.687360 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.687340    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d2827c94-1702-40e2-bb05-5fbef2c0d72b" (UID: "d2827c94-1702-40e2-bb05-5fbef2c0d72b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:42:40.687469 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.687428    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:40.687469 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.687448    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:40.687469 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.687462    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:40.687798 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.687481    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:40.687798 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.687793    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-data" (OuterVolumeSpecName: "istio-data") pod "d2827c94-1702-40e2-bb05-5fbef2c0d72b" (UID: "d2827c94-1702-40e2-bb05-5fbef2c0d72b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:40.689190 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.689165    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-kube-api-access-jbp54" (OuterVolumeSpecName: "kube-api-access-jbp54") pod "d2827c94-1702-40e2-bb05-5fbef2c0d72b" (UID: "d2827c94-1702-40e2-bb05-5fbef2c0d72b"). InnerVolumeSpecName "kube-api-access-jbp54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:40.689312 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.689239    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-token" (OuterVolumeSpecName: "istio-token") pod "d2827c94-1702-40e2-bb05-5fbef2c0d72b" (UID: "d2827c94-1702-40e2-bb05-5fbef2c0d72b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:40.689400 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.689373    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d2827c94-1702-40e2-bb05-5fbef2c0d72b" (UID: "d2827c94-1702-40e2-bb05-5fbef2c0d72b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:40.689467 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.689452    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d2827c94-1702-40e2-bb05-5fbef2c0d72b" (UID: "d2827c94-1702-40e2-bb05-5fbef2c0d72b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:42:40.788649 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.788625    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbp54\" (UniqueName: \"kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-kube-api-access-jbp54\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:40.788649 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.788645    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:40.788649 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.788654    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:40.788849 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.788662    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:40.788849 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:40.788670    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2827c94-1702-40e2-bb05-5fbef2c0d72b-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:41.197515 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.197480    2580 generic.go:356] "Generic (PLEG): container finished" podID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerID="f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182" exitCode=0
Jul 29 18:42:41.197692 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.197530    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr" event={"ID":"d2827c94-1702-40e2-bb05-5fbef2c0d72b","Type":"ContainerDied","Data":"f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182"}
Jul 29 18:42:41.197692 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.197541    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"
Jul 29 18:42:41.197692 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.197604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr" event={"ID":"d2827c94-1702-40e2-bb05-5fbef2c0d72b","Type":"ContainerDied","Data":"f209a10167c13ba92d95934dc2d177f1a33971d13ecaadb16f585c3f66191452"}
Jul 29 18:42:41.197692 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.197628    2580 scope.go:117] "RemoveContainer" containerID="f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182"
Jul 29 18:42:41.207097 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.207080    2580 scope.go:117] "RemoveContainer" containerID="d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05"
Jul 29 18:42:41.215286 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.215268    2580 scope.go:117] "RemoveContainer" containerID="f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182"
Jul 29 18:42:41.215581 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:42:41.215562    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182\": container with ID starting with f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182 not found: ID does not exist" containerID="f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182"
Jul 29 18:42:41.215628 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.215589    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182"} err="failed to get container status \"f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182\": rpc error: code = NotFound desc = could not find container \"f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182\": container with ID starting with f06cc49b47ecc1f550d2e90ab6371c25498ee07a8f0391656f122d67a9e21182 not found: ID does not exist"
Jul 29 18:42:41.215628 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.215608    2580 scope.go:117] "RemoveContainer" containerID="d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05"
Jul 29 18:42:41.215853 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:42:41.215835    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05\": container with ID starting with d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05 not found: ID does not exist" containerID="d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05"
Jul 29 18:42:41.215903 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.215867    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05"} err="failed to get container status \"d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05\": rpc error: code = NotFound desc = could not find container \"d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05\": container with ID starting with d588bb11abe0bc5c788cfd5e291af44d592151f2458348e520bdec0b91d84c05 not found: ID does not exist"
Jul 29 18:42:41.219501 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.219479    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"]
Jul 29 18:42:41.224587 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.224566    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-54bd5a-openshift-default-54fccf4cd4-4f9sr"]
Jul 29 18:42:41.273682 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:41.273655    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" path="/var/lib/kubelet/pods/d2827c94-1702-40e2-bb05-5fbef2c0d72b/volumes"
Jul 29 18:42:49.395106 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:49.395077    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:42:52.898761 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:52.898724    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"]
Jul 29 18:42:52.899273 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:52.898968    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy" containerID="cri-o://43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa" gracePeriod=30
Jul 29 18:42:53.787152 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.787122    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"]
Jul 29 18:42:53.787536 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.787523    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52a9cb5d-ae14-47d1-b567-7878744f7dd6" containerName="payload-processing"
Jul 29 18:42:53.787624 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.787538    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="52a9cb5d-ae14-47d1-b567-7878744f7dd6" containerName="payload-processing"
Jul 29 18:42:53.787624 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.787555    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerName="istio-proxy"
Jul 29 18:42:53.787624 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.787560    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerName="istio-proxy"
Jul 29 18:42:53.787624 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.787568    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerName="istio-proxy"
Jul 29 18:42:53.787624 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.787573    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerName="istio-proxy"
Jul 29 18:42:53.787892 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.787648    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerName="istio-proxy"
Jul 29 18:42:53.787892 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.787657    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2827c94-1702-40e2-bb05-5fbef2c0d72b" containerName="istio-proxy"
Jul 29 18:42:53.787892 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.787672    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="52a9cb5d-ae14-47d1-b567-7878744f7dd6" containerName="payload-processing"
Jul 29 18:42:53.793007 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.792988    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:53.795302 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.795284    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-109296-openshift-default-dockercfg-gp97r\""
Jul 29 18:42:53.802160 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.802138    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"]
Jul 29 18:42:53.905245 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.905210    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-envoy\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:53.905644 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.905310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-podinfo\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:53.905644 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.905357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-data\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:53.905644 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.905390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-certs\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:53.905644 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.905419    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-socket\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:53.905644 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.905470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istiod-ca-cert\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:53.905644 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.905518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-credential-socket\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:53.905644 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.905535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-token\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:53.905644 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:53.905549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c7g5\" (UniqueName: \"kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-kube-api-access-6c7g5\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.006676 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.006648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-socket\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.006676 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.006680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istiod-ca-cert\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.006904 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.006700    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-credential-socket\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.006904 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.006827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-token\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.006904 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.006866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c7g5\" (UniqueName: \"kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-kube-api-access-6c7g5\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.007109 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.006948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-envoy\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.007109 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.007047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-podinfo\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.007109 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.007072    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-socket\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.007271 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.007115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-data\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.007271 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.007119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-credential-socket\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.007271 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.007182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-certs\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.007471 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.007444    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-data\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.007471 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.007459    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istiod-ca-cert\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.007589 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.007471    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-certs\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.009390 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.009370    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-envoy\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.009610 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.009590    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-podinfo\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.014943 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.014923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-token\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.015059 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.014930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c7g5\" (UniqueName: \"kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-kube-api-access-6c7g5\") pod \"e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") " pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.105107 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.105033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:54.248444 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.248418    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"]
Jul 29 18:42:54.251131 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:42:54.251100    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde3ee8b_92f5_4621_bb40_4c387c6242e8.slice/crio-c811bfcad6c9e2071e5330377a15cf1addd06ea2d9755e5f689d3ef404c3861f WatchSource:0}: Error finding container c811bfcad6c9e2071e5330377a15cf1addd06ea2d9755e5f689d3ef404c3861f: Status 404 returned error can't find the container with id c811bfcad6c9e2071e5330377a15cf1addd06ea2d9755e5f689d3ef404c3861f
Jul 29 18:42:54.253906 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.253875    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:42:54.254012 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.253936    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:42:54.254012 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.253980    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:42:54.265009 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:54.264986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" event={"ID":"cde3ee8b-92f5-4621-bb40-4c387c6242e8","Type":"ContainerStarted","Data":"c811bfcad6c9e2071e5330377a15cf1addd06ea2d9755e5f689d3ef404c3861f"}
Jul 29 18:42:55.273741 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:55.273705    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" event={"ID":"cde3ee8b-92f5-4621-bb40-4c387c6242e8","Type":"ContainerStarted","Data":"39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b"}
Jul 29 18:42:55.290802 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:55.290756    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" podStartSLOduration=2.290740431 podStartE2EDuration="2.290740431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:42:55.289717853 +0000 UTC m=+3026.559216636" watchObservedRunningTime="2026-07-29 18:42:55.290740431 +0000 UTC m=+3026.560239214"
Jul 29 18:42:56.105483 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:56.105447    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:56.110283 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:56.110262    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:56.278019 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:56.277996    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:56.278751 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:56.278731    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:42:58.175061 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.174999    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:42:58.250705 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.250673    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-credential-socket\") pod \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") "
Jul 29 18:42:58.250837 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.250746    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istiod-ca-cert\") pod \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") "
Jul 29 18:42:58.250837 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.250784    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-envoy\") pod \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") "
Jul 29 18:42:58.250837 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.250805    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-podinfo\") pod \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") "
Jul 29 18:42:58.250837 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.250824    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "79373f51-e9bd-4a13-bc3b-ae6364aa3733" (UID: "79373f51-e9bd-4a13-bc3b-ae6364aa3733"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:58.251074 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.250877    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgg8v\" (UniqueName: \"kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-kube-api-access-hgg8v\") pod \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") "
Jul 29 18:42:58.251074 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.250906    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-data\") pod \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") "
Jul 29 18:42:58.251074 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.250934    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-certs\") pod \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") "
Jul 29 18:42:58.251074 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.251032    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-token\") pod \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") "
Jul 29 18:42:58.251074 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.251063    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-socket\") pod \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\" (UID: \"79373f51-e9bd-4a13-bc3b-ae6364aa3733\") "
Jul 29 18:42:58.251345 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.251256    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "79373f51-e9bd-4a13-bc3b-ae6364aa3733" (UID: "79373f51-e9bd-4a13-bc3b-ae6364aa3733"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:42:58.251435 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.251416    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:58.251507 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.251441    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:58.251507 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.251438    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "79373f51-e9bd-4a13-bc3b-ae6364aa3733" (UID: "79373f51-e9bd-4a13-bc3b-ae6364aa3733"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:58.251780 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.251705    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "79373f51-e9bd-4a13-bc3b-ae6364aa3733" (UID: "79373f51-e9bd-4a13-bc3b-ae6364aa3733"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:58.251920 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.251892    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-data" (OuterVolumeSpecName: "istio-data") pod "79373f51-e9bd-4a13-bc3b-ae6364aa3733" (UID: "79373f51-e9bd-4a13-bc3b-ae6364aa3733"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:58.253195 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.253168    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "79373f51-e9bd-4a13-bc3b-ae6364aa3733" (UID: "79373f51-e9bd-4a13-bc3b-ae6364aa3733"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:42:58.253300 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.253243    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-kube-api-access-hgg8v" (OuterVolumeSpecName: "kube-api-access-hgg8v") pod "79373f51-e9bd-4a13-bc3b-ae6364aa3733" (UID: "79373f51-e9bd-4a13-bc3b-ae6364aa3733"). InnerVolumeSpecName "kube-api-access-hgg8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:58.253300 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.253285    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "79373f51-e9bd-4a13-bc3b-ae6364aa3733" (UID: "79373f51-e9bd-4a13-bc3b-ae6364aa3733"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:42:58.253401 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.253387    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-token" (OuterVolumeSpecName: "istio-token") pod "79373f51-e9bd-4a13-bc3b-ae6364aa3733" (UID: "79373f51-e9bd-4a13-bc3b-ae6364aa3733"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:42:58.289438 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.289415    2580 generic.go:356] "Generic (PLEG): container finished" podID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerID="43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa" exitCode=0
Jul 29 18:42:58.289571 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.289482    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"
Jul 29 18:42:58.289571 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.289508    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" event={"ID":"79373f51-e9bd-4a13-bc3b-ae6364aa3733","Type":"ContainerDied","Data":"43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa"}
Jul 29 18:42:58.289571 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.289567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6" event={"ID":"79373f51-e9bd-4a13-bc3b-ae6364aa3733","Type":"ContainerDied","Data":"6a491c94d7a5c9844ddfff1fe3012fe4fb3c2d60e5e7a1a02b282bdd9aee75c8"}
Jul 29 18:42:58.289733 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.289590    2580 scope.go:117] "RemoveContainer" containerID="43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa"
Jul 29 18:42:58.298770 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.298757    2580 scope.go:117] "RemoveContainer" containerID="459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa"
Jul 29 18:42:58.307269 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.307252    2580 scope.go:117] "RemoveContainer" containerID="43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa"
Jul 29 18:42:58.307531 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:42:58.307511    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa\": container with ID starting with 43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa not found: ID does not exist" containerID="43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa"
Jul 29 18:42:58.307586 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.307540    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa"} err="failed to get container status \"43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa\": rpc error: code = NotFound desc = could not find container \"43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa\": container with ID starting with 43a785e33df46fe5ab1cd373108c249387def09e050ec5af5ab52fad3ec739fa not found: ID does not exist"
Jul 29 18:42:58.307586 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.307556    2580 scope.go:117] "RemoveContainer" containerID="459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa"
Jul 29 18:42:58.307792 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:42:58.307775    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa\": container with ID starting with 459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa not found: ID does not exist" containerID="459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa"
Jul 29 18:42:58.307838 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.307801    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa"} err="failed to get container status \"459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa\": rpc error: code = NotFound desc = could not find container \"459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa\": container with ID starting with 459da4fc86012ef647cb1bc99903a7553e40d1add7efd49b67563d68ade1caaa not found: ID does not exist"
Jul 29 18:42:58.312497 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.312474    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"]
Jul 29 18:42:58.317943 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.317924    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-084953-openshift-default-6587c9778-cx6b6"]
Jul 29 18:42:58.352251 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.352229    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgg8v\" (UniqueName: \"kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-kube-api-access-hgg8v\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:58.352251 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.352251    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:58.352399 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.352265    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:58.352399 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.352276    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:58.352399 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.352288    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:58.352399 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.352299    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:58.352399 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:58.352311    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79373f51-e9bd-4a13-bc3b-ae6364aa3733-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:42:59.274130 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:42:59.274100    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" path="/var/lib/kubelet/pods/79373f51-e9bd-4a13-bc3b-ae6364aa3733/volumes"
Jul 29 18:43:04.484978 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:04.484926    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:43:11.842477 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:11.842442    2580 patch_prober.go:28] interesting pod/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": read tcp 10.134.0.2:48602->10.134.0.49:15021: read: connection reset by peer" start-of-body=
Jul 29 18:43:11.842869 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:11.842547    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": read tcp 10.134.0.2:48602->10.134.0.49:15021: read: connection reset by peer"
Jul 29 18:43:12.362513 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:12.362480    2580 generic.go:356] "Generic (PLEG): container finished" podID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerID="39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b" exitCode=0
Jul 29 18:43:12.362513 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:12.362518    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" event={"ID":"cde3ee8b-92f5-4621-bb40-4c387c6242e8","Type":"ContainerDied","Data":"39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b"}
Jul 29 18:43:12.362946 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:12.362930    2580 scope.go:117] "RemoveContainer" containerID="39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b"
Jul 29 18:43:12.364010 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:12.363979    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:43:12.364094 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:12.364077    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:43:12.364135 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:12.364117    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:43:13.390646 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:13.390602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" event={"ID":"cde3ee8b-92f5-4621-bb40-4c387c6242e8","Type":"ContainerStarted","Data":"9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258"}
Jul 29 18:43:14.105435 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:14.105393    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:43:14.105629 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:14.105608    2580 patch_prober.go:28] interesting pod/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 18:43:14.105677 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:14.105658    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 18:43:15.106054 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:15.106015    2580 patch_prober.go:28] interesting pod/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 18:43:15.106460 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:15.106090    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 18:43:16.106323 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:16.106291    2580 patch_prober.go:28] interesting pod/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 18:43:16.106724 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:16.106345    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 18:43:17.105659 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:17.105620    2580 patch_prober.go:28] interesting pod/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 18:43:17.105859 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:17.105698    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 18:43:18.109161 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:18.109135    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:43:18.109629 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:18.109509    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:43:18.110212 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:18.110189    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:43:22.939935 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:22.939853    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"]
Jul 29 18:43:22.940633 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:22.940606    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy"
Jul 29 18:43:22.940633 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:22.940632    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy"
Jul 29 18:43:22.940855 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:22.940663    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy"
Jul 29 18:43:22.940855 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:22.940681    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy"
Jul 29 18:43:22.940855 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:22.940808    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy"
Jul 29 18:43:22.940855 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:22.940819    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="79373f51-e9bd-4a13-bc3b-ae6364aa3733" containerName="istio-proxy"
Jul 29 18:43:22.943456 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:22.943438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:22.945452 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:22.945428    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-95c5f7-openshift-default-dockercfg-dpk45\""
Jul 29 18:43:22.955700 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:22.955676    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"]
Jul 29 18:43:23.083621 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.083593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-credential-socket\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.083780 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.083633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-socket\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.083780 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.083663    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-certs\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.083780 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.083707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-podinfo\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.083780 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.083725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlrc4\" (UniqueName: \"kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-kube-api-access-tlrc4\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.083780 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.083774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-envoy\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.084077 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.083804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-data\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.084077 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.083824    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-token\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.084077 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.083920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istiod-ca-cert\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.185419 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.185383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-token\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.185585 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.185433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istiod-ca-cert\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.185585 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.185477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-credential-socket\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.185585 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.185516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-socket\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.185585 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.185557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-certs\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.185809 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.185595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-podinfo\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.185809 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.185622    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlrc4\" (UniqueName: \"kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-kube-api-access-tlrc4\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.185809 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.185681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-envoy\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.185809 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.185721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-data\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.186062 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.185998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-credential-socket\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.186062 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.186042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-certs\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.186172 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.186112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-data\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.186365 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.186344    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-socket\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.186432 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.186348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istiod-ca-cert\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.187998 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.187979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-envoy\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.188455 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.188432    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-podinfo\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.193915 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.193861    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-token\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.194219 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.194201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlrc4\" (UniqueName: \"kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-kube-api-access-tlrc4\") pod \"e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") " pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.257180 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.257158    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:23.402706 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.402678    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"]
Jul 29 18:43:23.405286 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:43:23.405252    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f3f67be_e0a1_4422_bfd7_c66fc08f78ab.slice/crio-f6c9a3076b9b0990b6c5de2e3dcb4ec1e3cf3adf278cb0bcb94d781d451a858f WatchSource:0}: Error finding container f6c9a3076b9b0990b6c5de2e3dcb4ec1e3cf3adf278cb0bcb94d781d451a858f: Status 404 returned error can't find the container with id f6c9a3076b9b0990b6c5de2e3dcb4ec1e3cf3adf278cb0bcb94d781d451a858f
Jul 29 18:43:23.407499 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.407463    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:43:23.407574 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.407554    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:43:23.407614 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.407601    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:43:23.446851 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:23.446797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" event={"ID":"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab","Type":"ContainerStarted","Data":"f6c9a3076b9b0990b6c5de2e3dcb4ec1e3cf3adf278cb0bcb94d781d451a858f"}
Jul 29 18:43:24.453170 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:24.453130    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" event={"ID":"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab","Type":"ContainerStarted","Data":"507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974"}
Jul 29 18:43:24.470916 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:24.470861    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" podStartSLOduration=2.470843553 podStartE2EDuration="2.470843553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:43:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:43:24.46902741 +0000 UTC m=+3055.738526196" watchObservedRunningTime="2026-07-29 18:43:24.470843553 +0000 UTC m=+3055.740342337"
Jul 29 18:43:25.257835 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:25.257785    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:25.262970 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:25.262926    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:25.458742 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:25.458713    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:25.459595 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:25.459571    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:32.502557 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:32.502523    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:43:40.589150 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:40.588780    2580 patch_prober.go:28] interesting pod/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:37712->10.134.0.50:15021: read: connection reset by peer" start-of-body=
Jul 29 18:43:40.589150 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:40.588840    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:37712->10.134.0.50:15021: read: connection reset by peer"
Jul 29 18:43:41.546111 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:41.546081    2580 generic.go:356] "Generic (PLEG): container finished" podID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerID="507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974" exitCode=0
Jul 29 18:43:41.546271 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:41.546157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" event={"ID":"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab","Type":"ContainerDied","Data":"507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974"}
Jul 29 18:43:41.546546 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:41.546532    2580 scope.go:117] "RemoveContainer" containerID="507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974"
Jul 29 18:43:41.547530 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:41.547502    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:43:41.547629 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:41.547588    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:43:41.547692 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:41.547634    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:43:42.552750 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:42.552714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" event={"ID":"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab","Type":"ContainerStarted","Data":"e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40"}
Jul 29 18:43:43.257766 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:43.257731    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:43.257973 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:43.257902    2580 patch_prober.go:28] interesting pod/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 29 18:43:43.258026 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:43.257982    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 29 18:43:44.257785 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:44.257745    2580 patch_prober.go:28] interesting pod/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 29 18:43:44.258212 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:44.257810    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 29 18:43:45.257807 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:45.257772    2580 patch_prober.go:28] interesting pod/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 29 18:43:45.258232 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:45.257824    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 29 18:43:46.276078 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:46.276046    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:46.276478 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:46.276312    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:43:46.277006 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:43:46.276988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:44:11.795948 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:11.795916    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:44:16.025213 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:16.025180    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"]
Jul 29 18:44:16.025617 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:16.025438    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy" containerID="cri-o://9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258" gracePeriod=30
Jul 29 18:44:18.112371 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:18.112328    2580 patch_prober.go:28] interesting pod/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 18:44:18.112763 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:18.112390    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 18:44:21.287075 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.287017    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:44:21.310829 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.310799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istiod-ca-cert\") pod \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") "
Jul 29 18:44:21.310829 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.310835    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-certs\") pod \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") "
Jul 29 18:44:21.311107 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.310885    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6c7g5\" (UniqueName: \"kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-kube-api-access-6c7g5\") pod \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") "
Jul 29 18:44:21.311107 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.310927    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-socket\") pod \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") "
Jul 29 18:44:21.311107 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.310982    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-podinfo\") pod \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") "
Jul 29 18:44:21.311107 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.311044    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-credential-socket\") pod \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") "
Jul 29 18:44:21.311331 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.311124    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-envoy\") pod \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") "
Jul 29 18:44:21.311331 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.311168    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-token\") pod \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") "
Jul 29 18:44:21.311331 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.311196    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-data\") pod \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\" (UID: \"cde3ee8b-92f5-4621-bb40-4c387c6242e8\") "
Jul 29 18:44:21.311331 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.311228    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cde3ee8b-92f5-4621-bb40-4c387c6242e8" (UID: "cde3ee8b-92f5-4621-bb40-4c387c6242e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:44:21.311331 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.311260    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cde3ee8b-92f5-4621-bb40-4c387c6242e8" (UID: "cde3ee8b-92f5-4621-bb40-4c387c6242e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:44:21.311331 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.311284    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cde3ee8b-92f5-4621-bb40-4c387c6242e8" (UID: "cde3ee8b-92f5-4621-bb40-4c387c6242e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:44:21.311642 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.311409    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cde3ee8b-92f5-4621-bb40-4c387c6242e8" (UID: "cde3ee8b-92f5-4621-bb40-4c387c6242e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:44:21.312062 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.311995    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:21.312062 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.312021    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:21.312062 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.312037    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:21.312062 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.312053    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:21.312368 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.312110    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "cde3ee8b-92f5-4621-bb40-4c387c6242e8" (UID: "cde3ee8b-92f5-4621-bb40-4c387c6242e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:44:21.313320 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.313298    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-kube-api-access-6c7g5" (OuterVolumeSpecName: "kube-api-access-6c7g5") pod "cde3ee8b-92f5-4621-bb40-4c387c6242e8" (UID: "cde3ee8b-92f5-4621-bb40-4c387c6242e8"). InnerVolumeSpecName "kube-api-access-6c7g5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:44:21.313801 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.313778    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cde3ee8b-92f5-4621-bb40-4c387c6242e8" (UID: "cde3ee8b-92f5-4621-bb40-4c387c6242e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:44:21.313895 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.313854    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "cde3ee8b-92f5-4621-bb40-4c387c6242e8" (UID: "cde3ee8b-92f5-4621-bb40-4c387c6242e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:44:21.314015 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.313943    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cde3ee8b-92f5-4621-bb40-4c387c6242e8" (UID: "cde3ee8b-92f5-4621-bb40-4c387c6242e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:44:21.412816 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.412762    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:21.412816 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.412807    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:21.412816 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.412819    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:21.412816 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.412830    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6c7g5\" (UniqueName: \"kubernetes.io/projected/cde3ee8b-92f5-4621-bb40-4c387c6242e8-kube-api-access-6c7g5\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:21.413170 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.412842    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cde3ee8b-92f5-4621-bb40-4c387c6242e8-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:21.750478 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.750385    2580 generic.go:356] "Generic (PLEG): container finished" podID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerID="9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258" exitCode=0
Jul 29 18:44:21.750478 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.750441    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"
Jul 29 18:44:21.750666 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.750471    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" event={"ID":"cde3ee8b-92f5-4621-bb40-4c387c6242e8","Type":"ContainerDied","Data":"9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258"}
Jul 29 18:44:21.750666 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.750545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5" event={"ID":"cde3ee8b-92f5-4621-bb40-4c387c6242e8","Type":"ContainerDied","Data":"c811bfcad6c9e2071e5330377a15cf1addd06ea2d9755e5f689d3ef404c3861f"}
Jul 29 18:44:21.750666 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.750571    2580 scope.go:117] "RemoveContainer" containerID="9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258"
Jul 29 18:44:21.759920 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.759904    2580 scope.go:117] "RemoveContainer" containerID="39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b"
Jul 29 18:44:21.768106 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.768093    2580 scope.go:117] "RemoveContainer" containerID="9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258"
Jul 29 18:44:21.768614 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:44:21.768586    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258\": container with ID starting with 9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258 not found: ID does not exist" containerID="9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258"
Jul 29 18:44:21.768733 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.768624    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258"} err="failed to get container status \"9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258\": rpc error: code = NotFound desc = could not find container \"9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258\": container with ID starting with 9599d6d44d2066e8a8676966f032d8ede8039bd9c05df0c195fb33791bdac258 not found: ID does not exist"
Jul 29 18:44:21.768733 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.768656    2580 scope.go:117] "RemoveContainer" containerID="39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b"
Jul 29 18:44:21.769206 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:44:21.769178    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b\": container with ID starting with 39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b not found: ID does not exist" containerID="39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b"
Jul 29 18:44:21.769319 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.769213    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b"} err="failed to get container status \"39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b\": rpc error: code = NotFound desc = could not find container \"39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b\": container with ID starting with 39f414f5ba13abd1dfe90ac8e3c90ad252bf35ad9ddce90773c53005d5c6235b not found: ID does not exist"
Jul 29 18:44:21.771225 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.771207    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"]
Jul 29 18:44:21.774175 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:21.774155    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-109296-openshift-default-7b5bcbf46-cj6k5"]
Jul 29 18:44:23.273252 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:23.273213    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" path="/var/lib/kubelet/pods/cde3ee8b-92f5-4621-bb40-4c387c6242e8/volumes"
Jul 29 18:44:36.291393 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:36.291360    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:44:38.449633 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:38.449592    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"]
Jul 29 18:44:38.450170 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:38.449847    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerName="istio-proxy" containerID="cri-o://e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40" gracePeriod=30
Jul 29 18:44:39.321205 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.321169    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"]
Jul 29 18:44:39.321673 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.321644    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy"
Jul 29 18:44:39.321673 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.321664    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy"
Jul 29 18:44:39.321673 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.321673    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy"
Jul 29 18:44:39.321673 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.321678    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy"
Jul 29 18:44:39.321933 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.321774    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy"
Jul 29 18:44:39.321933 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.321789    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde3ee8b-92f5-4621-bb40-4c387c6242e8" containerName="istio-proxy"
Jul 29 18:44:39.324151 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.324137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.326197 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.326177    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-6e9884-openshift-default-dockercfg-jpjpd\""
Jul 29 18:44:39.335701 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.335678    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"]
Jul 29 18:44:39.363638 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.363605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-podinfo\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.363779 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.363677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-socket\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.363779 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.363761    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fc7b\" (UniqueName: \"kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-kube-api-access-5fc7b\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.363889 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.363807    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-certs\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.363889 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.363843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-envoy\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.363889 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.363864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-credential-socket\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.364025 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.363948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e7fd0a-3a51-4e96-8358-708e100cd921-istiod-ca-cert\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.364067 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.364033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-data\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.364067 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.364052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-token\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.464938 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.464893    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-data\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.465448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.464945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-token\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.465448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.465027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-podinfo\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.465448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.465082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-socket\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.465448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.465116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fc7b\" (UniqueName: \"kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-kube-api-access-5fc7b\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.465448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.465166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-certs\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.465448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.465209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-envoy\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.465448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.465242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-credential-socket\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.465448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.465300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e7fd0a-3a51-4e96-8358-708e100cd921-istiod-ca-cert\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.465448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.465377    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-data\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.465448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.465442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-socket\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.466095 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.465521    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-certs\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.466095 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.465665    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-credential-socket\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.466095 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.466047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e7fd0a-3a51-4e96-8358-708e100cd921-istiod-ca-cert\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.467605 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.467579    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-envoy\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.468314 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.468300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-podinfo\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.473562 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.473533    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-token\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.473782 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.473762    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fc7b\" (UniqueName: \"kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-kube-api-access-5fc7b\") pod \"e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") " pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.635501 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.635405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:39.781893 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.781861    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"]
Jul 29 18:44:39.787606 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:44:39.787572    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e7fd0a_3a51_4e96_8358_708e100cd921.slice/crio-0048480e0b37deac0583c79fba2e4c2541fea6378aa1e7c95a2e41692af96259 WatchSource:0}: Error finding container 0048480e0b37deac0583c79fba2e4c2541fea6378aa1e7c95a2e41692af96259: Status 404 returned error can't find the container with id 0048480e0b37deac0583c79fba2e4c2541fea6378aa1e7c95a2e41692af96259
Jul 29 18:44:39.790921 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.790887    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:44:39.791035 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.791000    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:44:39.791035 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.791032    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:44:39.836923 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:39.836798    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" event={"ID":"44e7fd0a-3a51-4e96-8358-708e100cd921","Type":"ContainerStarted","Data":"0048480e0b37deac0583c79fba2e4c2541fea6378aa1e7c95a2e41692af96259"}
Jul 29 18:44:40.843641 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:40.843602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" event={"ID":"44e7fd0a-3a51-4e96-8358-708e100cd921","Type":"ContainerStarted","Data":"e328a0cb0ef5d850607b089fd38b05711eb612f3614c95e8c06f2acf6fd1335a"}
Jul 29 18:44:40.863431 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:40.863386    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" podStartSLOduration=1.863369995 podStartE2EDuration="1.863369995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:44:40.861336269 +0000 UTC m=+3132.130835053" watchObservedRunningTime="2026-07-29 18:44:40.863369995 +0000 UTC m=+3132.132868849"
Jul 29 18:44:41.636166 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:41.636128    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:41.641043 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:41.641014    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:41.849109 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:41.849077    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:41.849911 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:41.849891    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:43.708125 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.708103    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:44:43.807266 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807240    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-podinfo\") pod \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") "
Jul 29 18:44:43.807440 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807275    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-envoy\") pod \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") "
Jul 29 18:44:43.807440 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807299    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlrc4\" (UniqueName: \"kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-kube-api-access-tlrc4\") pod \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") "
Jul 29 18:44:43.807440 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807326    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-certs\") pod \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") "
Jul 29 18:44:43.807440 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807347    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istiod-ca-cert\") pod \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") "
Jul 29 18:44:43.807440 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807383    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-data\") pod \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") "
Jul 29 18:44:43.807440 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807415    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-token\") pod \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") "
Jul 29 18:44:43.807758 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807443    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-socket\") pod \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") "
Jul 29 18:44:43.807758 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807510    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-credential-socket\") pod \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\" (UID: \"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab\") "
Jul 29 18:44:43.807758 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807628    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" (UID: "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:44:43.807925 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807790    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" (UID: "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:44:43.807925 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807864    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" (UID: "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:44:43.807925 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807883    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:43.807925 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807903    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:43.808134 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.807943    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" (UID: "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:44:43.808264 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.808238    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" (UID: "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:44:43.810333 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.810304    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" (UID: "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:44:43.810426 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.810330    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" (UID: "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:44:43.810426 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.810415    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-kube-api-access-tlrc4" (OuterVolumeSpecName: "kube-api-access-tlrc4") pod "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" (UID: "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab"). InnerVolumeSpecName "kube-api-access-tlrc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:44:43.810506 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.810422    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" (UID: "5f3f67be-e0a1-4422-bfd7-c66fc08f78ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:44:43.861082 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.861056    2580 generic.go:356] "Generic (PLEG): container finished" podID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerID="e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40" exitCode=0
Jul 29 18:44:43.861176 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.861092    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" event={"ID":"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab","Type":"ContainerDied","Data":"e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40"}
Jul 29 18:44:43.861176 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.861114    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"
Jul 29 18:44:43.861176 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.861141    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb" event={"ID":"5f3f67be-e0a1-4422-bfd7-c66fc08f78ab","Type":"ContainerDied","Data":"f6c9a3076b9b0990b6c5de2e3dcb4ec1e3cf3adf278cb0bcb94d781d451a858f"}
Jul 29 18:44:43.861176 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.861161    2580 scope.go:117] "RemoveContainer" containerID="e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40"
Jul 29 18:44:43.871608 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.871594    2580 scope.go:117] "RemoveContainer" containerID="507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974"
Jul 29 18:44:43.879772 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.879754    2580 scope.go:117] "RemoveContainer" containerID="e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40"
Jul 29 18:44:43.880026 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:44:43.880009    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40\": container with ID starting with e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40 not found: ID does not exist" containerID="e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40"
Jul 29 18:44:43.880095 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.880033    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40"} err="failed to get container status \"e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40\": rpc error: code = NotFound desc = could not find container \"e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40\": container with ID starting with e3781a074d0e1a003ec03a26bb99342acc858168668f8f026c12269686b9cd40 not found: ID does not exist"
Jul 29 18:44:43.880095 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.880049    2580 scope.go:117] "RemoveContainer" containerID="507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974"
Jul 29 18:44:43.880279 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:44:43.880257    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974\": container with ID starting with 507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974 not found: ID does not exist" containerID="507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974"
Jul 29 18:44:43.880324 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.880286    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974"} err="failed to get container status \"507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974\": rpc error: code = NotFound desc = could not find container \"507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974\": container with ID starting with 507d80def55c78edea782652627e2008e7a9916cf6eee6610d168b739ceca974 not found: ID does not exist"
Jul 29 18:44:43.883200 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.883180    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"]
Jul 29 18:44:43.890155 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.890137    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-95c5f7-openshift-default-57b946ff84-mbptb"]
Jul 29 18:44:43.908359 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.908340    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:43.908420 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.908359    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:43.908420 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.908370    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:43.908420 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.908379    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:43.908420 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.908387    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlrc4\" (UniqueName: \"kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-kube-api-access-tlrc4\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:43.908420 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.908395    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:43.908420 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:43.908402    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:44:45.272584 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:45.272552    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" path="/var/lib/kubelet/pods/5f3f67be-e0a1-4422-bfd7-c66fc08f78ab/volumes"
Jul 29 18:44:47.491425 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:47.491389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:44:55.927933 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:55.927864    2580 generic.go:356] "Generic (PLEG): container finished" podID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerID="e328a0cb0ef5d850607b089fd38b05711eb612f3614c95e8c06f2acf6fd1335a" exitCode=0
Jul 29 18:44:55.927933 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:55.927903    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" event={"ID":"44e7fd0a-3a51-4e96-8358-708e100cd921","Type":"ContainerDied","Data":"e328a0cb0ef5d850607b089fd38b05711eb612f3614c95e8c06f2acf6fd1335a"}
Jul 29 18:44:55.928353 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:55.928259    2580 scope.go:117] "RemoveContainer" containerID="e328a0cb0ef5d850607b089fd38b05711eb612f3614c95e8c06f2acf6fd1335a"
Jul 29 18:44:55.929252 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:55.929222    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:44:55.929313 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:55.929294    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:44:55.929350 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:55.929324    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:44:56.848992 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:56.848938    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:56.934688 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:56.934650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" event={"ID":"44e7fd0a-3a51-4e96-8358-708e100cd921","Type":"ContainerStarted","Data":"f45eb56fd27c79411c02fc1aafbed12118fa97ed8c699aa76f310bcb3698bbdb"}
Jul 29 18:44:57.636153 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:57.636120    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:44:57.637314 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:57.637288    2580 patch_prober.go:28] interesting pod/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 18:44:57.637443 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:57.637348    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 18:44:58.636078 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:58.636045    2580 patch_prober.go:28] interesting pod/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 18:44:58.636441 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:58.636098    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 18:44:59.636303 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:59.636265    2580 patch_prober.go:28] interesting pod/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 18:44:59.636781 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:44:59.636329    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 18:45:00.636056 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:00.636019    2580 patch_prober.go:28] interesting pod/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 18:45:00.636228 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:00.636084    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 18:45:01.639877 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:01.639843    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:45:01.640263 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:01.640181    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:45:01.640895 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:01.640880    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 18:45:08.348366 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.348338    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"]
Jul 29 18:45:08.348749 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.348716    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerName="istio-proxy"
Jul 29 18:45:08.348749 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.348726    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerName="istio-proxy"
Jul 29 18:45:08.348823 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.348797    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerName="istio-proxy"
Jul 29 18:45:08.348823 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.348806    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerName="istio-proxy"
Jul 29 18:45:08.351401 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.351377    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.353376 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.353358    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-750f0d-openshift-default-dockercfg-xrvv4\""
Jul 29 18:45:08.362385 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.362362    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"]
Jul 29 18:45:08.447251 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.447225    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-envoy\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.447394 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.447278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c48mw\" (UniqueName: \"kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-kube-api-access-c48mw\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.447394 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.447348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-certs\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.447499 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.447395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-token\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.447499 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.447464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-credential-socket\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.447594 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.447515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-socket\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.447594 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.447535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istiod-ca-cert\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.447594 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.447558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-podinfo\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.447690 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.447611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-data\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.548361 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-certs\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.548515 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-token\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.548515 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-credential-socket\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.548515 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-socket\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.548515 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istiod-ca-cert\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.548515 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-podinfo\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.548515 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-data\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.548515 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-envoy\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.548863 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c48mw\" (UniqueName: \"kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-kube-api-access-c48mw\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.548863 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548762    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-certs\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.548863 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-credential-socket\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.549062 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.548888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-socket\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.549246 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.549213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-data\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.549319 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.549301    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istiod-ca-cert\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.550968 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.550934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-envoy\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.551196 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.551179    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-podinfo\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.555729 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.555709    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-token\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.555944 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.555927    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c48mw\" (UniqueName: \"kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-kube-api-access-c48mw\") pod \"e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") " pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.662109 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.662050    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:08.798140 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.798112    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"]
Jul 29 18:45:08.799839 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:45:08.799809    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod542f7807_2aa1_4ff7_8f74_7f052c8c3ab1.slice/crio-f4bd0efa0055d5f9fde4dd33b8dd17d0f14ba2559a359282152343ceb948275c WatchSource:0}: Error finding container f4bd0efa0055d5f9fde4dd33b8dd17d0f14ba2559a359282152343ceb948275c: Status 404 returned error can't find the container with id f4bd0efa0055d5f9fde4dd33b8dd17d0f14ba2559a359282152343ceb948275c
Jul 29 18:45:08.802256 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.802218    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:45:08.802330 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.802305    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:45:08.802377 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:08.802350    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:45:09.006289 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:09.006203    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" event={"ID":"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1","Type":"ContainerStarted","Data":"c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be"}
Jul 29 18:45:09.006289 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:09.006264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" event={"ID":"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1","Type":"ContainerStarted","Data":"f4bd0efa0055d5f9fde4dd33b8dd17d0f14ba2559a359282152343ceb948275c"}
Jul 29 18:45:09.025510 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:09.025458    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" podStartSLOduration=1.025443901 podStartE2EDuration="1.025443901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:45:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:45:09.022783145 +0000 UTC m=+3160.292281940" watchObservedRunningTime="2026-07-29 18:45:09.025443901 +0000 UTC m=+3160.294942683"
Jul 29 18:45:09.662798 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:09.662767    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:09.667426 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:09.667403    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:10.011273 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:10.011198    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:10.012087 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:10.012067    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:17.886687 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:17.886653    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:45:25.900577 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:25.900174    2580 patch_prober.go:28] interesting pod/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:47390->10.134.0.52:15021: read: connection reset by peer" start-of-body=
Jul 29 18:45:25.900577 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:25.900250    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": read tcp 10.134.0.2:47390->10.134.0.52:15021: read: connection reset by peer"
Jul 29 18:45:26.100162 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:26.100102    2580 generic.go:356] "Generic (PLEG): container finished" podID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerID="c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be" exitCode=0
Jul 29 18:45:26.100162 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:26.100155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" event={"ID":"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1","Type":"ContainerDied","Data":"c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be"}
Jul 29 18:45:26.100534 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:26.100518    2580 scope.go:117] "RemoveContainer" containerID="c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be"
Jul 29 18:45:26.101529 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:26.101501    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:45:26.101584 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:26.101571    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:45:26.101634 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:26.101601    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:45:27.107702 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:27.107665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" event={"ID":"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1","Type":"ContainerStarted","Data":"fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f"}
Jul 29 18:45:27.662577 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:27.662546    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:27.662768 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:27.662746    2580 patch_prober.go:28] interesting pod/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 29 18:45:27.662850 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:27.662802    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 29 18:45:28.662329 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:28.662296    2580 patch_prober.go:28] interesting pod/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 29 18:45:28.662788 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:28.662353    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 29 18:45:29.662614 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:29.662588    2580 patch_prober.go:28] interesting pod/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 29 18:45:29.662923 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:29.662641    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 29 18:45:30.663066 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:30.663030    2580 patch_prober.go:28] interesting pod/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 29 18:45:30.663625 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:30.663592    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 29 18:45:31.665637 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:31.665610    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:31.666101 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:31.665871    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:31.666501 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:31.666483    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 18:45:59.297594 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:45:59.297554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:46:21.724581 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.724501    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"]
Jul 29 18:46:21.725200 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.725182    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerName="istio-proxy"
Jul 29 18:46:21.725252 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.725204    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f3f67be-e0a1-4422-bfd7-c66fc08f78ab" containerName="istio-proxy"
Jul 29 18:46:21.728142 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.728125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.730900 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.730880    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"default-dockercfg-m2gfx\""
Jul 29 18:46:21.731042 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.730910    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"openshift-service-ca.crt\""
Jul 29 18:46:21.731042 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.730881    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"auth-test-model-750f0d-kserve-self-signed-certs\""
Jul 29 18:46:21.731139 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.731079    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"kube-root-ca.crt\""
Jul 29 18:46:21.738111 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.737682    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"]
Jul 29 18:46:21.823711 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.823678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-dshm\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.823882 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.823728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-home\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.823882 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.823776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-model-cache\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.823882 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.823811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tmp-dir\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.823882 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.823856    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tls-certs\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.824064 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.823888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjhrv\" (UniqueName: \"kubernetes.io/projected/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-kube-api-access-rjhrv\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.924594 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.924558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjhrv\" (UniqueName: \"kubernetes.io/projected/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-kube-api-access-rjhrv\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.924765 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.924604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-dshm\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.924765 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.924644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-home\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.924765 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.924665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-model-cache\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.924765 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.924697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tmp-dir\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.924765 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.924729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tls-certs\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.925103 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.925085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-home\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.925196 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.925180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-model-cache\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.925236 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.925204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tmp-dir\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.927097 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.927081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-dshm\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.927366 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.927350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tls-certs\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:21.932937 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:21.932915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjhrv\" (UniqueName: \"kubernetes.io/projected/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-kube-api-access-rjhrv\") pod \"auth-test-model-750f0d-kserve-57b6c59c95-4bf8r\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") " pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:22.040910 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:22.040877    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:22.175110 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:22.175050    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"]
Jul 29 18:46:22.177675 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:46:22.177649    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc773212_2a7e_4b92_a07a_ae9b5e133cb1.slice/crio-c68ce3b2e945725fcc55ec034bb6ee921079e5f0d2c6a48776c96c1a81475cd9 WatchSource:0}: Error finding container c68ce3b2e945725fcc55ec034bb6ee921079e5f0d2c6a48776c96c1a81475cd9: Status 404 returned error can't find the container with id c68ce3b2e945725fcc55ec034bb6ee921079e5f0d2c6a48776c96c1a81475cd9
Jul 29 18:46:22.393738 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:22.393655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r" event={"ID":"cc773212-2a7e-4b92-a07a-ae9b5e133cb1","Type":"ContainerStarted","Data":"2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137"}
Jul 29 18:46:22.393738 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:22.393694    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r" event={"ID":"cc773212-2a7e-4b92-a07a-ae9b5e133cb1","Type":"ContainerStarted","Data":"c68ce3b2e945725fcc55ec034bb6ee921079e5f0d2c6a48776c96c1a81475cd9"}
Jul 29 18:46:22.409745 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:22.409683    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r" podStartSLOduration=1.40966346 podStartE2EDuration="1.40966346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:46:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:46:22.40919734 +0000 UTC m=+3233.678696124" watchObservedRunningTime="2026-07-29 18:46:22.40966346 +0000 UTC m=+3233.679162245"
Jul 29 18:46:25.294876 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:25.294842    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:46:32.041294 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:32.041258    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:32.041752 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:32.041324    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:32.053793 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:32.053766    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:32.455069 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:32.454987    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:46:57.491736 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:46:57.491693    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:47:22.899936 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:47:22.899901    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:47:37.295562 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:47:37.295523    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:47:49.715781 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:47:49.715748    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:48:11.507226 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:48:11.507193    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:48:28.697573 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:48:28.697537    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:48:55.385855 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:48:55.385821    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:49:08.396730 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:49:08.396700    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:49:19.698810 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:49:19.698774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:49:39.392001 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:49:39.391910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:49:56.794154 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:49:56.794123    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:50:23.200264 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:50:23.200235    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:50:40.890771 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:50:40.890737    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:51:07.687309 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:07.687270    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:51:13.510543 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:13.510511    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"]
Jul 29 18:51:23.794798 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:23.794766    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:51:28.530692 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.530635    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r" podUID="cc773212-2a7e-4b92-a07a-ae9b5e133cb1" containerName="main" containerID="cri-o://2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137" gracePeriod=45
Jul 29 18:51:28.791316 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.791294    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:51:28.921177 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921148    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tls-certs\") pod \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") "
Jul 29 18:51:28.921338 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921194    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-model-cache\") pod \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") "
Jul 29 18:51:28.921338 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921227    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjhrv\" (UniqueName: \"kubernetes.io/projected/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-kube-api-access-rjhrv\") pod \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") "
Jul 29 18:51:28.921338 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921275    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-home\") pod \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") "
Jul 29 18:51:28.921338 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921297    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-dshm\") pod \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") "
Jul 29 18:51:28.921338 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921312    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tmp-dir\") pod \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\" (UID: \"cc773212-2a7e-4b92-a07a-ae9b5e133cb1\") "
Jul 29 18:51:28.921624 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921509    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-model-cache" (OuterVolumeSpecName: "model-cache") pod "cc773212-2a7e-4b92-a07a-ae9b5e133cb1" (UID: "cc773212-2a7e-4b92-a07a-ae9b5e133cb1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:51:28.921624 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921523    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-home" (OuterVolumeSpecName: "home") pod "cc773212-2a7e-4b92-a07a-ae9b5e133cb1" (UID: "cc773212-2a7e-4b92-a07a-ae9b5e133cb1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:51:28.921702 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921620    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cc773212-2a7e-4b92-a07a-ae9b5e133cb1" (UID: "cc773212-2a7e-4b92-a07a-ae9b5e133cb1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:51:28.921737 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921716    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-model-cache\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:51:28.921776 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921736    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-home\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:51:28.921776 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.921749    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tmp-dir\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:51:28.923582 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.923550    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-dshm" (OuterVolumeSpecName: "dshm") pod "cc773212-2a7e-4b92-a07a-ae9b5e133cb1" (UID: "cc773212-2a7e-4b92-a07a-ae9b5e133cb1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:51:28.923582 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.923561    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cc773212-2a7e-4b92-a07a-ae9b5e133cb1" (UID: "cc773212-2a7e-4b92-a07a-ae9b5e133cb1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:51:28.923722 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.923595    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-kube-api-access-rjhrv" (OuterVolumeSpecName: "kube-api-access-rjhrv") pod "cc773212-2a7e-4b92-a07a-ae9b5e133cb1" (UID: "cc773212-2a7e-4b92-a07a-ae9b5e133cb1"). InnerVolumeSpecName "kube-api-access-rjhrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:51:28.974576 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.974551    2580 generic.go:356] "Generic (PLEG): container finished" podID="cc773212-2a7e-4b92-a07a-ae9b5e133cb1" containerID="2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137" exitCode=0
Jul 29 18:51:28.974685 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.974594    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"
Jul 29 18:51:28.974685 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.974594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r" event={"ID":"cc773212-2a7e-4b92-a07a-ae9b5e133cb1","Type":"ContainerDied","Data":"2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137"}
Jul 29 18:51:28.974781 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.974686    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r" event={"ID":"cc773212-2a7e-4b92-a07a-ae9b5e133cb1","Type":"ContainerDied","Data":"c68ce3b2e945725fcc55ec034bb6ee921079e5f0d2c6a48776c96c1a81475cd9"}
Jul 29 18:51:28.974781 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.974703    2580 scope.go:117] "RemoveContainer" containerID="2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137"
Jul 29 18:51:28.983900 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.983884    2580 scope.go:117] "RemoveContainer" containerID="2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137"
Jul 29 18:51:28.984177 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:51:28.984155    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137\": container with ID starting with 2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137 not found: ID does not exist" containerID="2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137"
Jul 29 18:51:28.984250 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.984188    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137"} err="failed to get container status \"2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137\": rpc error: code = NotFound desc = could not find container \"2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137\": container with ID starting with 2cd3c3e33e3d8853e38f6e8bf816c680e1ac6d5222abe83c65becaceab16e137 not found: ID does not exist"
Jul 29 18:51:28.995810 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.995781    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"]
Jul 29 18:51:28.998312 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:28.998292    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/auth-test-model-750f0d-kserve-57b6c59c95-4bf8r"]
Jul 29 18:51:29.022669 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:29.022644    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjhrv\" (UniqueName: \"kubernetes.io/projected/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-kube-api-access-rjhrv\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:51:29.022669 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:29.022667    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-dshm\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:51:29.022794 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:29.022681    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc773212-2a7e-4b92-a07a-ae9b5e133cb1-tls-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:51:29.274350 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:29.274274    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc773212-2a7e-4b92-a07a-ae9b5e133cb1" path="/var/lib/kubelet/pods/cc773212-2a7e-4b92-a07a-ae9b5e133cb1/volumes"
Jul 29 18:51:33.595698 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:33.595664    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:51:56.878324 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.878289    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"]
Jul 29 18:51:56.878808 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.878747    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc773212-2a7e-4b92-a07a-ae9b5e133cb1" containerName="main"
Jul 29 18:51:56.878808 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.878759    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc773212-2a7e-4b92-a07a-ae9b5e133cb1" containerName="main"
Jul 29 18:51:56.878883 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.878840    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc773212-2a7e-4b92-a07a-ae9b5e133cb1" containerName="main"
Jul 29 18:51:56.882038 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.882016    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:56.886708 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.886687    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"openshift-service-ca.crt\""
Jul 29 18:51:56.886837 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.886708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"default-dockercfg-m2gfx\""
Jul 29 18:51:56.886837 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.886689    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"sub-test-model-750f0d-kserve-self-signed-certs\""
Jul 29 18:51:56.889517 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.889358    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-750f0d\"/\"kube-root-ca.crt\""
Jul 29 18:51:56.899810 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.899782    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"]
Jul 29 18:51:56.966636 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.966610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06e98036-666f-4a5f-845d-16ca2d028025-tls-certs\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:56.966760 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.966650    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gqdr\" (UniqueName: \"kubernetes.io/projected/06e98036-666f-4a5f-845d-16ca2d028025-kube-api-access-4gqdr\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:56.966760 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.966685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-tmp-dir\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:56.966760 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.966747    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-model-cache\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:56.966886 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.966819    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-dshm\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:56.966939 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:56.966924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-home\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.067663 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.067635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-dshm\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.067800 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.067722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-home\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.067800 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.067762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06e98036-666f-4a5f-845d-16ca2d028025-tls-certs\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.067800 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.067783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gqdr\" (UniqueName: \"kubernetes.io/projected/06e98036-666f-4a5f-845d-16ca2d028025-kube-api-access-4gqdr\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.067950 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.067816    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-tmp-dir\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.067950 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.067847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-model-cache\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.068237 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.068210    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-home\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.068354 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.068240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-model-cache\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.068354 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.068308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-tmp-dir\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.070148 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.070127    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-dshm\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.070437 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.070417    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06e98036-666f-4a5f-845d-16ca2d028025-tls-certs\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.075689 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.075669    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gqdr\" (UniqueName: \"kubernetes.io/projected/06e98036-666f-4a5f-845d-16ca2d028025-kube-api-access-4gqdr\") pod \"sub-test-model-750f0d-kserve-bd858f4c5-z6x5h\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") " pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.192020 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.191930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:51:57.331203 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:57.331178    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"]
Jul 29 18:51:57.333903 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:51:57.333873    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06e98036_666f_4a5f_845d_16ca2d028025.slice/crio-df9e47cff1c612760ac1628027d218d978a88990333d3b48e6ed8badf492eaa8 WatchSource:0}: Error finding container df9e47cff1c612760ac1628027d218d978a88990333d3b48e6ed8badf492eaa8: Status 404 returned error can't find the container with id df9e47cff1c612760ac1628027d218d978a88990333d3b48e6ed8badf492eaa8
Jul 29 18:51:58.117634 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:58.117600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h" event={"ID":"06e98036-666f-4a5f-845d-16ca2d028025","Type":"ContainerStarted","Data":"e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c"}
Jul 29 18:51:58.118019 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:58.117642    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h" event={"ID":"06e98036-666f-4a5f-845d-16ca2d028025","Type":"ContainerStarted","Data":"df9e47cff1c612760ac1628027d218d978a88990333d3b48e6ed8badf492eaa8"}
Jul 29 18:51:58.134579 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:51:58.134531    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h" podStartSLOduration=2.134515936 podStartE2EDuration="2.134515936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:51:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:51:58.132141014 +0000 UTC m=+3569.401639808" watchObservedRunningTime="2026-07-29 18:51:58.134515936 +0000 UTC m=+3569.404014719"
Jul 29 18:52:00.401301 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:52:00.401263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:52:07.192448 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:52:07.192416    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:52:07.192916 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:52:07.192467    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:52:07.204831 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:52:07.204808    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:52:08.179803 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:52:08.179776    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:52:47.895583 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:52:47.895547    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:53:06.398436 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:06.398392    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:53:28.293190 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:28.293154    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:53:34.942727 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:34.942691    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"]
Jul 29 18:53:45.488677 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:45.488641    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:53:49.962351 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:49.962310    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h" podUID="06e98036-666f-4a5f-845d-16ca2d028025" containerName="main" containerID="cri-o://e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c" gracePeriod=45
Jul 29 18:53:50.225438 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.225384    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:53:50.349904 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.349872    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-home\") pod \"06e98036-666f-4a5f-845d-16ca2d028025\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") "
Jul 29 18:53:50.349904 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.349902    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-dshm\") pod \"06e98036-666f-4a5f-845d-16ca2d028025\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") "
Jul 29 18:53:50.350106 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.349935    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06e98036-666f-4a5f-845d-16ca2d028025-tls-certs\") pod \"06e98036-666f-4a5f-845d-16ca2d028025\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") "
Jul 29 18:53:50.350106 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.349950    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-model-cache\") pod \"06e98036-666f-4a5f-845d-16ca2d028025\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") "
Jul 29 18:53:50.350106 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.350029    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-tmp-dir\") pod \"06e98036-666f-4a5f-845d-16ca2d028025\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") "
Jul 29 18:53:50.350106 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.350047    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gqdr\" (UniqueName: \"kubernetes.io/projected/06e98036-666f-4a5f-845d-16ca2d028025-kube-api-access-4gqdr\") pod \"06e98036-666f-4a5f-845d-16ca2d028025\" (UID: \"06e98036-666f-4a5f-845d-16ca2d028025\") "
Jul 29 18:53:50.350298 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.350203    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-home" (OuterVolumeSpecName: "home") pod "06e98036-666f-4a5f-845d-16ca2d028025" (UID: "06e98036-666f-4a5f-845d-16ca2d028025"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:53:50.350600 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.350313    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-model-cache" (OuterVolumeSpecName: "model-cache") pod "06e98036-666f-4a5f-845d-16ca2d028025" (UID: "06e98036-666f-4a5f-845d-16ca2d028025"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:53:50.350868 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.350312    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "06e98036-666f-4a5f-845d-16ca2d028025" (UID: "06e98036-666f-4a5f-845d-16ca2d028025"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:53:50.352173 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.352147    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-dshm" (OuterVolumeSpecName: "dshm") pod "06e98036-666f-4a5f-845d-16ca2d028025" (UID: "06e98036-666f-4a5f-845d-16ca2d028025"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:53:50.352272 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.352223    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06e98036-666f-4a5f-845d-16ca2d028025-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "06e98036-666f-4a5f-845d-16ca2d028025" (UID: "06e98036-666f-4a5f-845d-16ca2d028025"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 18:53:50.352272 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.352250    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06e98036-666f-4a5f-845d-16ca2d028025-kube-api-access-4gqdr" (OuterVolumeSpecName: "kube-api-access-4gqdr") pod "06e98036-666f-4a5f-845d-16ca2d028025" (UID: "06e98036-666f-4a5f-845d-16ca2d028025"). InnerVolumeSpecName "kube-api-access-4gqdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:53:50.450804 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.450775    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-tmp-dir\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:53:50.450804 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.450795    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gqdr\" (UniqueName: \"kubernetes.io/projected/06e98036-666f-4a5f-845d-16ca2d028025-kube-api-access-4gqdr\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:53:50.450804 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.450804    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-home\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:53:50.451075 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.450813    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-dshm\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:53:50.451075 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.450821    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06e98036-666f-4a5f-845d-16ca2d028025-tls-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:53:50.451075 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.450828    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06e98036-666f-4a5f-845d-16ca2d028025-model-cache\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:53:50.691808 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.691775    2580 generic.go:356] "Generic (PLEG): container finished" podID="06e98036-666f-4a5f-845d-16ca2d028025" containerID="e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c" exitCode=0
Jul 29 18:53:50.691940 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.691822    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"
Jul 29 18:53:50.691940 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.691867    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h" event={"ID":"06e98036-666f-4a5f-845d-16ca2d028025","Type":"ContainerDied","Data":"e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c"}
Jul 29 18:53:50.691940 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.691911    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h" event={"ID":"06e98036-666f-4a5f-845d-16ca2d028025","Type":"ContainerDied","Data":"df9e47cff1c612760ac1628027d218d978a88990333d3b48e6ed8badf492eaa8"}
Jul 29 18:53:50.691940 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.691932    2580 scope.go:117] "RemoveContainer" containerID="e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c"
Jul 29 18:53:50.701405 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.701387    2580 scope.go:117] "RemoveContainer" containerID="e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c"
Jul 29 18:53:50.701659 ip-10-0-129-228 kubenswrapper[2580]: E0729 18:53:50.701641    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c\": container with ID starting with e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c not found: ID does not exist" containerID="e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c"
Jul 29 18:53:50.701705 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.701668    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c"} err="failed to get container status \"e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c\": rpc error: code = NotFound desc = could not find container \"e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c\": container with ID starting with e935e440a965f8066df5fabaa0ee700b4674f32fe26a3808d926b995626cdc0c not found: ID does not exist"
Jul 29 18:53:50.710276 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.710253    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"]
Jul 29 18:53:50.713577 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:50.713557    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-750f0d/sub-test-model-750f0d-kserve-bd858f4c5-z6x5h"]
Jul 29 18:53:51.282078 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:51.282044    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06e98036-666f-4a5f-845d-16ca2d028025" path="/var/lib/kubelet/pods/06e98036-666f-4a5f-845d-16ca2d028025/volumes"
Jul 29 18:53:54.591935 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:53:54.591899    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:54:25.900345 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:54:25.900309    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:55:12.827000 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:55:12.826967    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:55:32.299037 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:55:32.299006    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:55:55.090833 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:55:55.090797    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:56:09.194605 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:09.194569    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:56:34.699318 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:34.699274    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:56:52.132788 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.132679    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"]
Jul 29 18:56:52.133436 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.133413    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06e98036-666f-4a5f-845d-16ca2d028025" containerName="main"
Jul 29 18:56:52.133502 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.133440    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="06e98036-666f-4a5f-845d-16ca2d028025" containerName="main"
Jul 29 18:56:52.133556 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.133542    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="06e98036-666f-4a5f-845d-16ca2d028025" containerName="main"
Jul 29 18:56:52.136929 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.136902    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.138988 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.138969    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-a10ee3-openshift-default-dockercfg-rhlnl\""
Jul 29 18:56:52.145512 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.145484    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"]
Jul 29 18:56:52.268332 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.268289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-token\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.268537 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.268360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istiod-ca-cert\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.268537 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.268434    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-certs\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.268537 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.268500    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-envoy\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.268537 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.268529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-podinfo\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.268767 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.268589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzspj\" (UniqueName: \"kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-kube-api-access-mzspj\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.268767 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.268698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-credential-socket\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.268767 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.268728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-data\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.268767 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.268764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-socket\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.369475 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.369439    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzspj\" (UniqueName: \"kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-kube-api-access-mzspj\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.369673 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.369514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-credential-socket\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.369673 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.369532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-data\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.369673 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.369557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-socket\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.369673 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.369588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-token\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.369673 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.369627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istiod-ca-cert\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.369673 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.369652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-certs\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.370040 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.369684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-envoy\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.370040 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.369704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-podinfo\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.370040 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.370026    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-credential-socket\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.370198 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.370074    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-socket\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.370198 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.370104    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-data\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.370314 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.370272    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-certs\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.370603 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.370576    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istiod-ca-cert\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.372291 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.372272    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-podinfo\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.372381 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.372339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-envoy\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.377422 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.377395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-token\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.377575 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.377557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzspj\" (UniqueName: \"kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-kube-api-access-mzspj\") pod \"e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") " pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.449150 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.449062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:52.591790 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.591762    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"]
Jul 29 18:56:52.593398 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:56:52.593371    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode48a3423_4778_4fb5_9384_2ac7dc90a48d.slice/crio-d3f5e455e4b193f1c80665aebce18c607d26e4ee587773b8103171a756d70bbb WatchSource:0}: Error finding container d3f5e455e4b193f1c80665aebce18c607d26e4ee587773b8103171a756d70bbb: Status 404 returned error can't find the container with id d3f5e455e4b193f1c80665aebce18c607d26e4ee587773b8103171a756d70bbb
Jul 29 18:56:52.595563 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.595530    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:56:52.595642 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.595600    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:56:52.595642 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.595629    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:56:52.607286 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:52.607258    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" event={"ID":"e48a3423-4778-4fb5-9384-2ac7dc90a48d","Type":"ContainerStarted","Data":"d3f5e455e4b193f1c80665aebce18c607d26e4ee587773b8103171a756d70bbb"}
Jul 29 18:56:53.614112 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:53.614077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" event={"ID":"e48a3423-4778-4fb5-9384-2ac7dc90a48d","Type":"ContainerStarted","Data":"618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa"}
Jul 29 18:56:53.632939 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:53.632885    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" podStartSLOduration=1.632866683 podStartE2EDuration="1.632866683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:56:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:56:53.632295686 +0000 UTC m=+3864.901794468" watchObservedRunningTime="2026-07-29 18:56:53.632866683 +0000 UTC m=+3864.902365469"
Jul 29 18:56:54.449943 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:54.449909    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:54.454477 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:54.454453    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:54.619708 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:54.619679    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:54.620605 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:54.620585    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:56:56.324624 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.324591    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"]
Jul 29 18:56:56.328426 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.328401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 18:56:56.330358 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.330337    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-a10ee3\""
Jul 29 18:56:56.337368 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.337346    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"]
Jul 29 18:56:56.406375 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.406347    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ce7788-4842-491e-ad18-43c688114c24-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf\" (UID: \"56ce7788-4842-491e-ad18-43c688114c24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 18:56:56.406507 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.406464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44dp7\" (UniqueName: \"kubernetes.io/projected/56ce7788-4842-491e-ad18-43c688114c24-kube-api-access-44dp7\") pod \"payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf\" (UID: \"56ce7788-4842-491e-ad18-43c688114c24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 18:56:56.507434 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.507392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44dp7\" (UniqueName: \"kubernetes.io/projected/56ce7788-4842-491e-ad18-43c688114c24-kube-api-access-44dp7\") pod \"payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf\" (UID: \"56ce7788-4842-491e-ad18-43c688114c24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 18:56:56.507635 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.507561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ce7788-4842-491e-ad18-43c688114c24-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf\" (UID: \"56ce7788-4842-491e-ad18-43c688114c24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 18:56:56.508308 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.508286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ce7788-4842-491e-ad18-43c688114c24-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf\" (UID: \"56ce7788-4842-491e-ad18-43c688114c24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 18:56:56.516419 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.516395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44dp7\" (UniqueName: \"kubernetes.io/projected/56ce7788-4842-491e-ad18-43c688114c24-kube-api-access-44dp7\") pod \"payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf\" (UID: \"56ce7788-4842-491e-ad18-43c688114c24\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 18:56:56.640718 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.640660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 18:56:56.777626 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:56.777602    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"]
Jul 29 18:56:56.780311 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:56:56.780284    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56ce7788_4842_491e_ad18_43c688114c24.slice/crio-a2e95aebb499c74a40d77c4bfca8e065a62c73234f865175076007a1c26f3198 WatchSource:0}: Error finding container a2e95aebb499c74a40d77c4bfca8e065a62c73234f865175076007a1c26f3198: Status 404 returned error can't find the container with id a2e95aebb499c74a40d77c4bfca8e065a62c73234f865175076007a1c26f3198
Jul 29 18:56:57.636786 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:57.636747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf" event={"ID":"56ce7788-4842-491e-ad18-43c688114c24","Type":"ContainerStarted","Data":"0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c"}
Jul 29 18:56:57.637262 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:57.636799    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf" event={"ID":"56ce7788-4842-491e-ad18-43c688114c24","Type":"ContainerStarted","Data":"a2e95aebb499c74a40d77c4bfca8e065a62c73234f865175076007a1c26f3198"}
Jul 29 18:56:57.637262 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:57.636839    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 18:56:57.654477 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:56:57.654442    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf" podStartSLOduration=1.654428807 podStartE2EDuration="1.654428807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:56:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:56:57.653069732 +0000 UTC m=+3868.922568514" watchObservedRunningTime="2026-07-29 18:56:57.654428807 +0000 UTC m=+3868.923927567"
Jul 29 18:57:04.596396 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:04.596361    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:57:08.650588 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:08.650556    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 18:57:13.732642 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:13.732605    2580 generic.go:356] "Generic (PLEG): container finished" podID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerID="618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa" exitCode=0
Jul 29 18:57:13.732642 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:13.732644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" event={"ID":"e48a3423-4778-4fb5-9384-2ac7dc90a48d","Type":"ContainerDied","Data":"618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa"}
Jul 29 18:57:13.733181 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:13.732947    2580 scope.go:117] "RemoveContainer" containerID="618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa"
Jul 29 18:57:13.734006 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:13.733976    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:57:13.734113 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:13.734041    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:57:13.734113 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:13.734081    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:57:14.739380 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:14.739344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" event={"ID":"e48a3423-4778-4fb5-9384-2ac7dc90a48d","Type":"ContainerStarted","Data":"f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0"}
Jul 29 18:57:15.450209 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:15.450172    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:57:15.450438 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:15.450413    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 29 18:57:15.450485 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:15.450469    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 29 18:57:16.450432 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:16.450399    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 29 18:57:16.450806 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:16.450449    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 29 18:57:17.449617 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:17.449581    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 29 18:57:17.449786 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:17.449637    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 29 18:57:18.450033 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:18.449991    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 29 18:57:18.450424 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:18.450048    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 29 18:57:19.453343 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:19.453313    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:57:19.453736 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:19.453718    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:57:19.454363 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:19.454346    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 18:57:35.797534 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:57:35.797494    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:58:01.697677 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:01.697642    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:58:12.959576 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.959544    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"]
Jul 29 18:58:12.962593 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.962575    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:12.964790 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.964766    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-8c76b3-openshift-default-dockercfg-w8bnw\""
Jul 29 18:58:12.973876 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.973851    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"]
Jul 29 18:58:12.997167 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.997142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-data\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:12.997321 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.997170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-podinfo\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:12.997321 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.997202    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:12.997321 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.997271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-envoy\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:12.997499 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.997335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n28pq\" (UniqueName: \"kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-kube-api-access-n28pq\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:12.997499 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.997364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-token\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:12.997499 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.997464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-socket\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:12.997499 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.997487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-credential-socket\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:12.997713 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:12.997516    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-certs\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.098248 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n28pq\" (UniqueName: \"kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-kube-api-access-n28pq\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.098248 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-token\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.098504 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098320    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-socket\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.098504 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-credential-socket\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.098504 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-certs\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.098504 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-data\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.098504 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-podinfo\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.098504 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.098504 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-envoy\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.098871 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-data\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.098871 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098819    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-socket\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.099004 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098859    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-credential-socket\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.099004 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.098900    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-certs\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.099221 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.099202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.100861 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.100835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-envoy\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.101037 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.101017    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-podinfo\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.106245 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.106225    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-token\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.106359 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.106271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n28pq\" (UniqueName: \"kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-kube-api-access-n28pq\") pod \"e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") " pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.274810 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.274752    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:13.425392 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.425362    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"]
Jul 29 18:58:13.427060 ip-10-0-129-228 kubenswrapper[2580]: W0729 18:58:13.427033    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb3b54d1_f91e_4c5b_90e6_c5b524807dc9.slice/crio-1f838cf11b917030188ac60058c648e4deb5c9918633775d4553b14d6f69f162 WatchSource:0}: Error finding container 1f838cf11b917030188ac60058c648e4deb5c9918633775d4553b14d6f69f162: Status 404 returned error can't find the container with id 1f838cf11b917030188ac60058c648e4deb5c9918633775d4553b14d6f69f162
Jul 29 18:58:13.429400 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.429365    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:58:13.429491 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.429432    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:58:13.429491 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:13.429462    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:58:14.053703 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:14.053666    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" event={"ID":"db3b54d1-f91e-4c5b-90e6-c5b524807dc9","Type":"ContainerStarted","Data":"e529ab8674f892ff8fcb74c6f8c39511d88be452f6b0cede238412bcf3dcc99d"}
Jul 29 18:58:14.054096 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:14.053718    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" event={"ID":"db3b54d1-f91e-4c5b-90e6-c5b524807dc9","Type":"ContainerStarted","Data":"1f838cf11b917030188ac60058c648e4deb5c9918633775d4553b14d6f69f162"}
Jul 29 18:58:14.073875 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:14.073830    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" podStartSLOduration=2.073814989 podStartE2EDuration="2.073814989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 18:58:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 18:58:14.072041146 +0000 UTC m=+3945.341539933" watchObservedRunningTime="2026-07-29 18:58:14.073814989 +0000 UTC m=+3945.343313847"
Jul 29 18:58:14.275128 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:14.275098    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:14.276243 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:14.276221    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 29 18:58:14.276303 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:14.276271    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 29 18:58:15.279808 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:15.279773    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:16.065166 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:16.065131    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:16.066005 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:16.065988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:21.396085 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:21.396013    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:58:30.141948 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:30.141912    2580 generic.go:356] "Generic (PLEG): container finished" podID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerID="e529ab8674f892ff8fcb74c6f8c39511d88be452f6b0cede238412bcf3dcc99d" exitCode=0
Jul 29 18:58:30.142368 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:30.141989    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" event={"ID":"db3b54d1-f91e-4c5b-90e6-c5b524807dc9","Type":"ContainerDied","Data":"e529ab8674f892ff8fcb74c6f8c39511d88be452f6b0cede238412bcf3dcc99d"}
Jul 29 18:58:30.142368 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:30.142360    2580 scope.go:117] "RemoveContainer" containerID="e529ab8674f892ff8fcb74c6f8c39511d88be452f6b0cede238412bcf3dcc99d"
Jul 29 18:58:30.143431 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:30.143404    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:58:30.143484 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:30.143470    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:58:30.143527 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:30.143518    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 18:58:31.065476 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:31.065443    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:31.149154 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:31.149119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" event={"ID":"db3b54d1-f91e-4c5b-90e6-c5b524807dc9","Type":"ContainerStarted","Data":"9fe9580bff6be9b856fb05f62fe2156e68565ccda5eb39b8f18350392775d3a2"}
Jul 29 18:58:31.277752 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:31.277704    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 29 18:58:31.277977 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:31.277774    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 29 18:58:31.282249 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:31.282218    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:32.276067 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:32.276021    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 29 18:58:32.276710 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:32.276677    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 29 18:58:33.275908 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:33.275875    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 29 18:58:33.276197 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:33.276170    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 29 18:58:34.275918 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:34.275874    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 29 18:58:34.276147 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:34.275936    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 29 18:58:35.279456 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:35.279427    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:35.279831 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:35.279711    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:35.280405 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:35.280383    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:50.795438 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:50.795403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:58:53.030346 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:53.030311    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"]
Jul 29 18:58:53.031212 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:53.031151    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy" containerID="cri-o://9fe9580bff6be9b856fb05f62fe2156e68565ccda5eb39b8f18350392775d3a2" gracePeriod=30
Jul 29 18:58:58.293850 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.293819    2580 generic.go:356] "Generic (PLEG): container finished" podID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerID="9fe9580bff6be9b856fb05f62fe2156e68565ccda5eb39b8f18350392775d3a2" exitCode=0
Jul 29 18:58:58.294264 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.293890    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" event={"ID":"db3b54d1-f91e-4c5b-90e6-c5b524807dc9","Type":"ContainerDied","Data":"9fe9580bff6be9b856fb05f62fe2156e68565ccda5eb39b8f18350392775d3a2"}
Jul 29 18:58:58.294264 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.293984    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn" event={"ID":"db3b54d1-f91e-4c5b-90e6-c5b524807dc9","Type":"ContainerDied","Data":"1f838cf11b917030188ac60058c648e4deb5c9918633775d4553b14d6f69f162"}
Jul 29 18:58:58.294264 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.294014    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f838cf11b917030188ac60058c648e4deb5c9918633775d4553b14d6f69f162"
Jul 29 18:58:58.294264 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.294034    2580 scope.go:117] "RemoveContainer" containerID="e529ab8674f892ff8fcb74c6f8c39511d88be452f6b0cede238412bcf3dcc99d"
Jul 29 18:58:58.296790 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.296762    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:58.327023 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.326998    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-credential-socket\") pod \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") "
Jul 29 18:58:58.327143 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.327038    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n28pq\" (UniqueName: \"kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-kube-api-access-n28pq\") pod \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") "
Jul 29 18:58:58.327143 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.327081    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-socket\") pod \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") "
Jul 29 18:58:58.327143 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.327109    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-token\") pod \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") "
Jul 29 18:58:58.327143 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.327124    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istiod-ca-cert\") pod \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") "
Jul 29 18:58:58.327373 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.327174    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-podinfo\") pod \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") "
Jul 29 18:58:58.327373 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.327289    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db3b54d1-f91e-4c5b-90e6-c5b524807dc9" (UID: "db3b54d1-f91e-4c5b-90e6-c5b524807dc9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:58:58.327473 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.327446    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db3b54d1-f91e-4c5b-90e6-c5b524807dc9" (UID: "db3b54d1-f91e-4c5b-90e6-c5b524807dc9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:58:58.327536 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.327511    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db3b54d1-f91e-4c5b-90e6-c5b524807dc9" (UID: "db3b54d1-f91e-4c5b-90e6-c5b524807dc9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 18:58:58.327620 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.327605    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-certs\") pod \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") "
Jul 29 18:58:58.327674 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.327653    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-data\") pod \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") "
Jul 29 18:58:58.327728 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.327679    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-envoy\") pod \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\" (UID: \"db3b54d1-f91e-4c5b-90e6-c5b524807dc9\") "
Jul 29 18:58:58.328137 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.328081    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:58:58.328137 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.328102    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:58:58.328137 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.328116    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:58:58.328137 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.328115    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db3b54d1-f91e-4c5b-90e6-c5b524807dc9" (UID: "db3b54d1-f91e-4c5b-90e6-c5b524807dc9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:58:58.328617 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.328596    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-data" (OuterVolumeSpecName: "istio-data") pod "db3b54d1-f91e-4c5b-90e6-c5b524807dc9" (UID: "db3b54d1-f91e-4c5b-90e6-c5b524807dc9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:58:58.329777 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.329743    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db3b54d1-f91e-4c5b-90e6-c5b524807dc9" (UID: "db3b54d1-f91e-4c5b-90e6-c5b524807dc9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 18:58:58.329777 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.329764    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-token" (OuterVolumeSpecName: "istio-token") pod "db3b54d1-f91e-4c5b-90e6-c5b524807dc9" (UID: "db3b54d1-f91e-4c5b-90e6-c5b524807dc9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:58:58.329946 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.329856    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-kube-api-access-n28pq" (OuterVolumeSpecName: "kube-api-access-n28pq") pod "db3b54d1-f91e-4c5b-90e6-c5b524807dc9" (UID: "db3b54d1-f91e-4c5b-90e6-c5b524807dc9"). InnerVolumeSpecName "kube-api-access-n28pq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 18:58:58.330043 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.330021    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db3b54d1-f91e-4c5b-90e6-c5b524807dc9" (UID: "db3b54d1-f91e-4c5b-90e6-c5b524807dc9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 18:58:58.429533 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.429509    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:58:58.429533 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.429530    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:58:58.429661 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.429541    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n28pq\" (UniqueName: \"kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-kube-api-access-n28pq\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:58:58.429661 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.429550    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:58:58.429661 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.429558    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:58:58.429661 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:58.429566    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db3b54d1-f91e-4c5b-90e6-c5b524807dc9-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 18:58:59.300764 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:59.300735    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"
Jul 29 18:58:59.331216 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:59.331187    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"]
Jul 29 18:58:59.337128 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:58:59.337105    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-8c76b3-openshift-default-7758458bb-lmlvn"]
Jul 29 18:59:01.273183 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:59:01.273147    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" path="/var/lib/kubelet/pods/db3b54d1-f91e-4c5b-90e6-c5b524807dc9/volumes"
Jul 29 18:59:11.194507 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:59:11.194473    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 18:59:21.791495 ip-10-0-129-228 kubenswrapper[2580]: I0729 18:59:21.791458    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 19:00:33.860408 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:33.860375    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"]
Jul 29 19:00:33.860933 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:33.860629    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf" podUID="56ce7788-4842-491e-ad18-43c688114c24" containerName="payload-pre-processing" containerID="cri-o://0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c" gracePeriod=30
Jul 29 19:00:34.128194 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.128169    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 19:00:34.224441 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.224415    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ce7788-4842-491e-ad18-43c688114c24-plugins-config-volume\") pod \"56ce7788-4842-491e-ad18-43c688114c24\" (UID: \"56ce7788-4842-491e-ad18-43c688114c24\") "
Jul 29 19:00:34.224612 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.224451    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44dp7\" (UniqueName: \"kubernetes.io/projected/56ce7788-4842-491e-ad18-43c688114c24-kube-api-access-44dp7\") pod \"56ce7788-4842-491e-ad18-43c688114c24\" (UID: \"56ce7788-4842-491e-ad18-43c688114c24\") "
Jul 29 19:00:34.224808 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.224784    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56ce7788-4842-491e-ad18-43c688114c24-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56ce7788-4842-491e-ad18-43c688114c24" (UID: "56ce7788-4842-491e-ad18-43c688114c24"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:00:34.226658 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.226637    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56ce7788-4842-491e-ad18-43c688114c24-kube-api-access-44dp7" (OuterVolumeSpecName: "kube-api-access-44dp7") pod "56ce7788-4842-491e-ad18-43c688114c24" (UID: "56ce7788-4842-491e-ad18-43c688114c24"). InnerVolumeSpecName "kube-api-access-44dp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:00:34.326155 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.326124    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56ce7788-4842-491e-ad18-43c688114c24-plugins-config-volume\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:00:34.326155 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.326151    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44dp7\" (UniqueName: \"kubernetes.io/projected/56ce7788-4842-491e-ad18-43c688114c24-kube-api-access-44dp7\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:00:34.807923 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.807890    2580 generic.go:356] "Generic (PLEG): container finished" podID="56ce7788-4842-491e-ad18-43c688114c24" containerID="0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c" exitCode=0
Jul 29 19:00:34.808148 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.807938    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"
Jul 29 19:00:34.808148 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.807989    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf" event={"ID":"56ce7788-4842-491e-ad18-43c688114c24","Type":"ContainerDied","Data":"0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c"}
Jul 29 19:00:34.808148 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.808058    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf" event={"ID":"56ce7788-4842-491e-ad18-43c688114c24","Type":"ContainerDied","Data":"a2e95aebb499c74a40d77c4bfca8e065a62c73234f865175076007a1c26f3198"}
Jul 29 19:00:34.808148 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.808089    2580 scope.go:117] "RemoveContainer" containerID="0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c"
Jul 29 19:00:34.817267 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.817252    2580 scope.go:117] "RemoveContainer" containerID="0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c"
Jul 29 19:00:34.817526 ip-10-0-129-228 kubenswrapper[2580]: E0729 19:00:34.817511    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c\": container with ID starting with 0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c not found: ID does not exist" containerID="0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c"
Jul 29 19:00:34.817569 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.817535    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c"} err="failed to get container status \"0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c\": rpc error: code = NotFound desc = could not find container \"0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c\": container with ID starting with 0d866933f503043f357458c7232ebc8e48cf8d766e46839f1cc26c4f595b5e7c not found: ID does not exist"
Jul 29 19:00:34.830015 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.829987    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"]
Jul 29 19:00:34.835660 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:34.835638    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a10ee3-84cbb57bfb-sf7jf"]
Jul 29 19:00:35.272983 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:35.272891    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56ce7788-4842-491e-ad18-43c688114c24" path="/var/lib/kubelet/pods/56ce7788-4842-491e-ad18-43c688114c24/volumes"
Jul 29 19:00:37.109011 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:37.108978    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 19:00:40.345725 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:40.345689    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"]
Jul 29 19:00:40.346148 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:40.345974    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerName="istio-proxy" containerID="cri-o://f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0" gracePeriod=30
Jul 29 19:00:45.614321 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.614252    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 19:00:45.740606 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.740570    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-certs\") pod \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") "
Jul 29 19:00:45.740811 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.740642    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzspj\" (UniqueName: \"kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-kube-api-access-mzspj\") pod \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") "
Jul 29 19:00:45.740811 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.740681    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-envoy\") pod \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") "
Jul 29 19:00:45.740811 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.740717    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-credential-socket\") pod \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") "
Jul 29 19:00:45.740811 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.740789    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-token\") pod \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") "
Jul 29 19:00:45.741068 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.740834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istiod-ca-cert\") pod \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") "
Jul 29 19:00:45.741068 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.740863    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-socket\") pod \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") "
Jul 29 19:00:45.741068 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.740867    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e48a3423-4778-4fb5-9384-2ac7dc90a48d" (UID: "e48a3423-4778-4fb5-9384-2ac7dc90a48d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:00:45.741068 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.740912    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-podinfo\") pod \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") "
Jul 29 19:00:45.741068 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.740949    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-data\") pod \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\" (UID: \"e48a3423-4778-4fb5-9384-2ac7dc90a48d\") "
Jul 29 19:00:45.741346 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.741074    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e48a3423-4778-4fb5-9384-2ac7dc90a48d" (UID: "e48a3423-4778-4fb5-9384-2ac7dc90a48d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:00:45.741346 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.741155    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e48a3423-4778-4fb5-9384-2ac7dc90a48d" (UID: "e48a3423-4778-4fb5-9384-2ac7dc90a48d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:00:45.741346 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.741322    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e48a3423-4778-4fb5-9384-2ac7dc90a48d" (UID: "e48a3423-4778-4fb5-9384-2ac7dc90a48d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:00:45.741495 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.741385    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:00:45.741495 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.741404    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:00:45.741495 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.741419    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:00:45.741986 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.741942    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-data" (OuterVolumeSpecName: "istio-data") pod "e48a3423-4778-4fb5-9384-2ac7dc90a48d" (UID: "e48a3423-4778-4fb5-9384-2ac7dc90a48d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:00:45.743036 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.743010    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-kube-api-access-mzspj" (OuterVolumeSpecName: "kube-api-access-mzspj") pod "e48a3423-4778-4fb5-9384-2ac7dc90a48d" (UID: "e48a3423-4778-4fb5-9384-2ac7dc90a48d"). InnerVolumeSpecName "kube-api-access-mzspj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:00:45.743266 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.743250    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e48a3423-4778-4fb5-9384-2ac7dc90a48d" (UID: "e48a3423-4778-4fb5-9384-2ac7dc90a48d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 19:00:45.743527 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.743508    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e48a3423-4778-4fb5-9384-2ac7dc90a48d" (UID: "e48a3423-4778-4fb5-9384-2ac7dc90a48d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:00:45.743600 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.743569    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e48a3423-4778-4fb5-9384-2ac7dc90a48d" (UID: "e48a3423-4778-4fb5-9384-2ac7dc90a48d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:00:45.842761 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.842726    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:00:45.842761 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.842755    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:00:45.842761 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.842764    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:00:45.843019 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.842774    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzspj\" (UniqueName: \"kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-kube-api-access-mzspj\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:00:45.843019 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.842785    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:00:45.843019 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.842792    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e48a3423-4778-4fb5-9384-2ac7dc90a48d-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:00:45.867905 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.867826    2580 generic.go:356] "Generic (PLEG): container finished" podID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerID="f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0" exitCode=0
Jul 29 19:00:45.867905 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.867878    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"
Jul 29 19:00:45.868143 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.867915    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" event={"ID":"e48a3423-4778-4fb5-9384-2ac7dc90a48d","Type":"ContainerDied","Data":"f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0"}
Jul 29 19:00:45.868143 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.868005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh" event={"ID":"e48a3423-4778-4fb5-9384-2ac7dc90a48d","Type":"ContainerDied","Data":"d3f5e455e4b193f1c80665aebce18c607d26e4ee587773b8103171a756d70bbb"}
Jul 29 19:00:45.868143 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.868040    2580 scope.go:117] "RemoveContainer" containerID="f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0"
Jul 29 19:00:45.879523 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.879492    2580 scope.go:117] "RemoveContainer" containerID="618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa"
Jul 29 19:00:45.888358 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.888342    2580 scope.go:117] "RemoveContainer" containerID="f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0"
Jul 29 19:00:45.888628 ip-10-0-129-228 kubenswrapper[2580]: E0729 19:00:45.888602    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0\": container with ID starting with f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0 not found: ID does not exist" containerID="f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0"
Jul 29 19:00:45.888749 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.888633    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0"} err="failed to get container status \"f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0\": rpc error: code = NotFound desc = could not find container \"f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0\": container with ID starting with f7ce0740052a1163eaa85c42a14ee1d184db586ff2cd622e944e37e067cde6e0 not found: ID does not exist"
Jul 29 19:00:45.888749 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.888659    2580 scope.go:117] "RemoveContainer" containerID="618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa"
Jul 29 19:00:45.888990 ip-10-0-129-228 kubenswrapper[2580]: E0729 19:00:45.888901    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa\": container with ID starting with 618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa not found: ID does not exist" containerID="618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa"
Jul 29 19:00:45.888990 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.888931    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa"} err="failed to get container status \"618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa\": rpc error: code = NotFound desc = could not find container \"618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa\": container with ID starting with 618cdc6a4a7561b7148c0a08807613f3811037f35f34fccd8e6859ca4179d1aa not found: ID does not exist"
Jul 29 19:00:45.891015 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.890993    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"]
Jul 29 19:00:45.897475 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:45.897455    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-a10ee3-openshift-default-75547fb966-d2njh"]
Jul 29 19:00:47.274051 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:47.274020    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" path="/var/lib/kubelet/pods/e48a3423-4778-4fb5-9384-2ac7dc90a48d/volumes"
Jul 29 19:00:54.398983 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:54.395403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 19:00:57.721499 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:57.721459    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"]
Jul 29 19:00:57.721945 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:00:57.721744    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy" containerID="cri-o://f45eb56fd27c79411c02fc1aafbed12118fa97ed8c699aa76f310bcb3698bbdb" gracePeriod=30
Jul 29 19:01:01.642598 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:01.642550    2580 patch_prober.go:28] interesting pod/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 19:01:01.642993 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:01.642645    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 19:01:02.963239 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:02.963201    2580 generic.go:356] "Generic (PLEG): container finished" podID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerID="f45eb56fd27c79411c02fc1aafbed12118fa97ed8c699aa76f310bcb3698bbdb" exitCode=0
Jul 29 19:01:02.963641 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:02.963279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" event={"ID":"44e7fd0a-3a51-4e96-8358-708e100cd921","Type":"ContainerDied","Data":"f45eb56fd27c79411c02fc1aafbed12118fa97ed8c699aa76f310bcb3698bbdb"}
Jul 29 19:01:02.963641 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:02.963357    2580 scope.go:117] "RemoveContainer" containerID="e328a0cb0ef5d850607b089fd38b05711eb612f3614c95e8c06f2acf6fd1335a"
Jul 29 19:01:02.996208 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:02.996153    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 19:01:03.111288 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111257    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fc7b\" (UniqueName: \"kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-kube-api-access-5fc7b\") pod \"44e7fd0a-3a51-4e96-8358-708e100cd921\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") "
Jul 29 19:01:03.111470 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111299    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-podinfo\") pod \"44e7fd0a-3a51-4e96-8358-708e100cd921\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") "
Jul 29 19:01:03.111470 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111317    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-data\") pod \"44e7fd0a-3a51-4e96-8358-708e100cd921\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") "
Jul 29 19:01:03.111470 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111342    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e7fd0a-3a51-4e96-8358-708e100cd921-istiod-ca-cert\") pod \"44e7fd0a-3a51-4e96-8358-708e100cd921\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") "
Jul 29 19:01:03.111470 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111394    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-certs\") pod \"44e7fd0a-3a51-4e96-8358-708e100cd921\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") "
Jul 29 19:01:03.111470 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111411    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-socket\") pod \"44e7fd0a-3a51-4e96-8358-708e100cd921\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") "
Jul 29 19:01:03.111470 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111462    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-token\") pod \"44e7fd0a-3a51-4e96-8358-708e100cd921\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") "
Jul 29 19:01:03.111793 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111518    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-envoy\") pod \"44e7fd0a-3a51-4e96-8358-708e100cd921\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") "
Jul 29 19:01:03.111793 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111549    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-credential-socket\") pod \"44e7fd0a-3a51-4e96-8358-708e100cd921\" (UID: \"44e7fd0a-3a51-4e96-8358-708e100cd921\") "
Jul 29 19:01:03.111793 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111732    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "44e7fd0a-3a51-4e96-8358-708e100cd921" (UID: "44e7fd0a-3a51-4e96-8358-708e100cd921"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:01:03.111793 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111725    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "44e7fd0a-3a51-4e96-8358-708e100cd921" (UID: "44e7fd0a-3a51-4e96-8358-708e100cd921"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:01:03.112057 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111833    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44e7fd0a-3a51-4e96-8358-708e100cd921-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "44e7fd0a-3a51-4e96-8358-708e100cd921" (UID: "44e7fd0a-3a51-4e96-8358-708e100cd921"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:01:03.112057 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111920    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:03.112057 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111941    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:03.112057 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.111981    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "44e7fd0a-3a51-4e96-8358-708e100cd921" (UID: "44e7fd0a-3a51-4e96-8358-708e100cd921"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:01:03.112404 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.112373    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-data" (OuterVolumeSpecName: "istio-data") pod "44e7fd0a-3a51-4e96-8358-708e100cd921" (UID: "44e7fd0a-3a51-4e96-8358-708e100cd921"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:01:03.113785 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.113761    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-kube-api-access-5fc7b" (OuterVolumeSpecName: "kube-api-access-5fc7b") pod "44e7fd0a-3a51-4e96-8358-708e100cd921" (UID: "44e7fd0a-3a51-4e96-8358-708e100cd921"). InnerVolumeSpecName "kube-api-access-5fc7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:01:03.113887 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.113811    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-token" (OuterVolumeSpecName: "istio-token") pod "44e7fd0a-3a51-4e96-8358-708e100cd921" (UID: "44e7fd0a-3a51-4e96-8358-708e100cd921"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:01:03.113887 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.113829    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "44e7fd0a-3a51-4e96-8358-708e100cd921" (UID: "44e7fd0a-3a51-4e96-8358-708e100cd921"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 19:01:03.114065 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.114050    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "44e7fd0a-3a51-4e96-8358-708e100cd921" (UID: "44e7fd0a-3a51-4e96-8358-708e100cd921"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:01:03.213291 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.213260    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:03.213291 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.213284    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:03.213291 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.213292    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:03.213507 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.213302    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fc7b\" (UniqueName: \"kubernetes.io/projected/44e7fd0a-3a51-4e96-8358-708e100cd921-kube-api-access-5fc7b\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:03.213507 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.213311    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:03.213507 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.213319    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e7fd0a-3a51-4e96-8358-708e100cd921-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:03.213507 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.213327    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e7fd0a-3a51-4e96-8358-708e100cd921-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:03.969824 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.969796    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"
Jul 29 19:01:03.970244 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.969849    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8" event={"ID":"44e7fd0a-3a51-4e96-8358-708e100cd921","Type":"ContainerDied","Data":"0048480e0b37deac0583c79fba2e4c2541fea6378aa1e7c95a2e41692af96259"}
Jul 29 19:01:03.970244 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.969905    2580 scope.go:117] "RemoveContainer" containerID="f45eb56fd27c79411c02fc1aafbed12118fa97ed8c699aa76f310bcb3698bbdb"
Jul 29 19:01:03.985797 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.985773    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"]
Jul 29 19:01:03.991302 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:03.991278    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-6e9884-openshift-default-7644bdcb4d-9knk8"]
Jul 29 19:01:05.272815 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:05.272774    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" path="/var/lib/kubelet/pods/44e7fd0a-3a51-4e96-8358-708e100cd921/volumes"
Jul 29 19:01:11.590081 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:11.590041    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-w4rbc"]
Jul 29 19:01:15.095663 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:15.095633    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"]
Jul 29 19:01:15.096153 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:15.095914    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy" containerID="cri-o://fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f" gracePeriod=30
Jul 29 19:01:16.668652 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:16.668606    2580 patch_prober.go:28] interesting pod/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 19:01:16.669196 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:16.668671    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 19:01:20.356923 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.356900    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 19:01:20.467030 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.466941    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-data\") pod \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") "
Jul 29 19:01:20.467030 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467005    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c48mw\" (UniqueName: \"kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-kube-api-access-c48mw\") pod \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") "
Jul 29 19:01:20.467230 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467032    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-credential-socket\") pod \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") "
Jul 29 19:01:20.467230 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467054    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-socket\") pod \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") "
Jul 29 19:01:20.467230 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467099    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-token\") pod \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") "
Jul 29 19:01:20.467230 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467124    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istiod-ca-cert\") pod \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") "
Jul 29 19:01:20.467230 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467155    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-certs\") pod \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") "
Jul 29 19:01:20.467230 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467195    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-podinfo\") pod \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") "
Jul 29 19:01:20.467230 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467220    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-envoy\") pod \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\" (UID: \"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1\") "
Jul 29 19:01:20.467563 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467355    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" (UID: "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:01:20.467563 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467379    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" (UID: "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:01:20.467673 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467562    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-credential-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:20.467673 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467603    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-socket\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:20.467673 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467648    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" (UID: "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:01:20.467904 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467746    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-data" (OuterVolumeSpecName: "istio-data") pod "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" (UID: "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:01:20.467992 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.467927    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" (UID: "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:01:20.469415 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.469385    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-kube-api-access-c48mw" (OuterVolumeSpecName: "kube-api-access-c48mw") pod "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" (UID: "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1"). InnerVolumeSpecName "kube-api-access-c48mw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:01:20.469522 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.469482    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" (UID: "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:01:20.469860 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.469836    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" (UID: "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 19:01:20.469860 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.469847    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-token" (OuterVolumeSpecName: "istio-token") pod "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" (UID: "542f7807-2aa1-4ff7-8f74-7f052c8c3ab1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:01:20.569002 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.568978    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istiod-ca-cert\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:20.569002 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.568999    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-workload-certs\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:20.569171 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.569010    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-podinfo\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:20.569171 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.569017    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-envoy\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:20.569171 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.569026    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-data\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:20.569171 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.569034    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c48mw\" (UniqueName: \"kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-kube-api-access-c48mw\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:20.569171 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.569044    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1-istio-token\") on node \"ip-10-0-129-228.ec2.internal\" DevicePath \"\""
Jul 29 19:01:20.946693 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:20.946666    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jkjr6_a71e79b4-cd7a-4d57-947e-fdbd388cbd60/manager/0.log"
Jul 29 19:01:21.063005 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.062949    2580 generic.go:356] "Generic (PLEG): container finished" podID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerID="fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f" exitCode=0
Jul 29 19:01:21.063168 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.063036    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"
Jul 29 19:01:21.063168 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.062985    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" event={"ID":"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1","Type":"ContainerDied","Data":"fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f"}
Jul 29 19:01:21.063294 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.063201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6" event={"ID":"542f7807-2aa1-4ff7-8f74-7f052c8c3ab1","Type":"ContainerDied","Data":"f4bd0efa0055d5f9fde4dd33b8dd17d0f14ba2559a359282152343ceb948275c"}
Jul 29 19:01:21.063294 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.063244    2580 scope.go:117] "RemoveContainer" containerID="fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f"
Jul 29 19:01:21.068078 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.068057    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xldbf_2897d9aa-2f97-46da-b23f-d7cc6cb28589/manager/0.log"
Jul 29 19:01:21.073633 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.073617    2580 scope.go:117] "RemoveContainer" containerID="c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be"
Jul 29 19:01:21.082233 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.082218    2580 scope.go:117] "RemoveContainer" containerID="fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f"
Jul 29 19:01:21.082475 ip-10-0-129-228 kubenswrapper[2580]: E0729 19:01:21.082451    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f\": container with ID starting with fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f not found: ID does not exist" containerID="fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f"
Jul 29 19:01:21.082559 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.082488    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f"} err="failed to get container status \"fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f\": rpc error: code = NotFound desc = could not find container \"fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f\": container with ID starting with fcbc305637db3feffa1b0d4abb895ddfc040676534da636070710296e584a96f not found: ID does not exist"
Jul 29 19:01:21.082559 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.082513    2580 scope.go:117] "RemoveContainer" containerID="c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be"
Jul 29 19:01:21.082938 ip-10-0-129-228 kubenswrapper[2580]: E0729 19:01:21.082912    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be\": container with ID starting with c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be not found: ID does not exist" containerID="c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be"
Jul 29 19:01:21.083048 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.082948    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be"} err="failed to get container status \"c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be\": rpc error: code = NotFound desc = could not find container \"c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be\": container with ID starting with c960229006afd2f3ca625915c6a75fe7bd7df75e8adc2ceaa262d23bd7b939be not found: ID does not exist"
Jul 29 19:01:21.085030 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.085012    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"]
Jul 29 19:01:21.090780 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.090761    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-750f0d-openshift-default-5c588ff84d-kbrv6"]
Jul 29 19:01:21.273477 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.273428    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" path="/var/lib/kubelet/pods/542f7807-2aa1-4ff7-8f74-7f052c8c3ab1/volumes"
Jul 29 19:01:21.545395 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.545374    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-d4fkp_620f475b-5a57-46e4-abe0-265834f558ac/copy-manifests/0.log"
Jul 29 19:01:21.551170 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:21.551147    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-d4fkp_620f475b-5a57-46e4-abe0-265834f558ac/manager/0.log"
Jul 29 19:01:23.955281 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:23.955254    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kshv2_a60a43e0-cd16-436d-bdb3-e22eb7130b9d/manager/0.log"
Jul 29 19:01:24.315316 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:24.315286    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4hv7l_d8c03dea-6afd-45fc-9540-aa3ceca004a9/registry-server/0.log"
Jul 29 19:01:24.541072 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:24.541042    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-w4rbc_fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80/limitador/0.log"
Jul 29 19:01:24.658247 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:24.658176    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9l6vp_5993b350-bae8-43a3-9c36-48a9a5ff9019/manager/0.log"
Jul 29 19:01:26.376619 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:26.376591    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-7dqqd_1821c52f-9b11-4525-84e5-e1407bf1fec9/main/0.log"
Jul 29 19:01:26.598418 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:26.598339    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-n46nx_f6875645-0f4d-457a-8947-de8b8d309e09/main/0.log"
Jul 29 19:01:35.257981 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:35.257917    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xmd4d_45f70f28-b309-4c10-ad6b-39ebcc5abcfc/global-pull-secret-syncer/0.log"
Jul 29 19:01:35.406812 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:35.406786    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nf779_19dd92a6-e163-466e-862e-0c79aebda97b/konnectivity-agent/0.log"
Jul 29 19:01:35.442008 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:35.441981    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-228.ec2.internal_d14432099af8323069e39bbc1aa22244/haproxy/0.log"
Jul 29 19:01:41.326409 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:41.326379    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kshv2_a60a43e0-cd16-436d-bdb3-e22eb7130b9d/manager/0.log"
Jul 29 19:01:41.453571 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:41.453539    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4hv7l_d8c03dea-6afd-45fc-9540-aa3ceca004a9/registry-server/0.log"
Jul 29 19:01:41.594617 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:41.594546    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-w4rbc_fef656ba-b5c3-4bc7-bf9e-5b21d96d0b80/limitador/0.log"
Jul 29 19:01:41.719295 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:41.719261    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9l6vp_5993b350-bae8-43a3-9c36-48a9a5ff9019/manager/0.log"
Jul 29 19:01:43.373447 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.373409    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6410a58-7371-4e28-ab42-6a03a5daaf71/alertmanager/0.log"
Jul 29 19:01:43.394098 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.394069    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6410a58-7371-4e28-ab42-6a03a5daaf71/config-reloader/0.log"
Jul 29 19:01:43.415120 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.415091    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6410a58-7371-4e28-ab42-6a03a5daaf71/kube-rbac-proxy-web/0.log"
Jul 29 19:01:43.432767 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.432747    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6410a58-7371-4e28-ab42-6a03a5daaf71/kube-rbac-proxy/0.log"
Jul 29 19:01:43.452522 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.452504    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6410a58-7371-4e28-ab42-6a03a5daaf71/kube-rbac-proxy-metric/0.log"
Jul 29 19:01:43.472404 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.472378    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6410a58-7371-4e28-ab42-6a03a5daaf71/prom-label-proxy/0.log"
Jul 29 19:01:43.490799 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.490783    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6410a58-7371-4e28-ab42-6a03a5daaf71/init-config-reloader/0.log"
Jul 29 19:01:43.569591 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.569570    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-9zfml_6c3280d6-4482-492a-8b20-879d7659b158/kube-state-metrics/0.log"
Jul 29 19:01:43.583638 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.583618    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-9zfml_6c3280d6-4482-492a-8b20-879d7659b158/kube-rbac-proxy-main/0.log"
Jul 29 19:01:43.606313 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.606295    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-9zfml_6c3280d6-4482-492a-8b20-879d7659b158/kube-rbac-proxy-self/0.log"
Jul 29 19:01:43.634546 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.634481    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-796596f7bf-nmj46_dacbd3e9-ab5a-43cd-9f67-1b382c6df7a9/metrics-server/0.log"
Jul 29 19:01:43.659305 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.659287    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-qd8n6_7e0d1e83-ce11-46fb-9ac5-74f9f7f61f37/monitoring-plugin/0.log"
Jul 29 19:01:43.854232 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.854210    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xh745_17c7fa6d-3d5d-4903-b359-6bdf57d1200e/node-exporter/0.log"
Jul 29 19:01:43.871249 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.871220    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xh745_17c7fa6d-3d5d-4903-b359-6bdf57d1200e/kube-rbac-proxy/0.log"
Jul 29 19:01:43.889344 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.889288    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xh745_17c7fa6d-3d5d-4903-b359-6bdf57d1200e/init-textfile/0.log"
Jul 29 19:01:43.919835 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.919817    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-62dx6_9b344506-3162-458d-b604-ae29eef14fef/kube-rbac-proxy-main/0.log"
Jul 29 19:01:43.944505 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.944478    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-62dx6_9b344506-3162-458d-b604-ae29eef14fef/kube-rbac-proxy-self/0.log"
Jul 29 19:01:43.964615 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.964598    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-62dx6_9b344506-3162-458d-b604-ae29eef14fef/openshift-state-metrics/0.log"
Jul 29 19:01:43.998411 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:43.998392    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf/prometheus/0.log"
Jul 29 19:01:44.016381 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.016363    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf/config-reloader/0.log"
Jul 29 19:01:44.035014 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.034996    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf/thanos-sidecar/0.log"
Jul 29 19:01:44.053487 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.053470    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf/kube-rbac-proxy-web/0.log"
Jul 29 19:01:44.073179 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.073156    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf/kube-rbac-proxy/0.log"
Jul 29 19:01:44.091037 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.091018    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf/kube-rbac-proxy-thanos/0.log"
Jul 29 19:01:44.109776 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.109748    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cdedfc2b-e5e7-41a0-bdf9-85a8bd42e1bf/init-config-reloader/0.log"
Jul 29 19:01:44.139204 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.139173    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-m5xkx_495dc79f-7292-4d2f-bfc6-203eb16368d3/prometheus-operator/0.log"
Jul 29 19:01:44.155146 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.155090    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-m5xkx_495dc79f-7292-4d2f-bfc6-203eb16368d3/kube-rbac-proxy/0.log"
Jul 29 19:01:44.212063 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.212032    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85445b56b5-j9ww2_ef621e30-667c-4e22-b48d-a5aaf92d3a09/telemeter-client/0.log"
Jul 29 19:01:44.227244 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.227226    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85445b56b5-j9ww2_ef621e30-667c-4e22-b48d-a5aaf92d3a09/reload/0.log"
Jul 29 19:01:44.245812 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.245795    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85445b56b5-j9ww2_ef621e30-667c-4e22-b48d-a5aaf92d3a09/kube-rbac-proxy/0.log"
Jul 29 19:01:44.273495 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.273476    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c75594f9-44g4v_eb63a03f-7d69-4af8-81ac-0b80a5eb9adf/thanos-query/0.log"
Jul 29 19:01:44.304060 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.304040    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c75594f9-44g4v_eb63a03f-7d69-4af8-81ac-0b80a5eb9adf/kube-rbac-proxy-web/0.log"
Jul 29 19:01:44.322689 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.322672    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c75594f9-44g4v_eb63a03f-7d69-4af8-81ac-0b80a5eb9adf/kube-rbac-proxy/0.log"
Jul 29 19:01:44.340140 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.340124    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c75594f9-44g4v_eb63a03f-7d69-4af8-81ac-0b80a5eb9adf/prom-label-proxy/0.log"
Jul 29 19:01:44.359816 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.359792    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c75594f9-44g4v_eb63a03f-7d69-4af8-81ac-0b80a5eb9adf/kube-rbac-proxy-rules/0.log"
Jul 29 19:01:44.377354 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:44.377334    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84c75594f9-44g4v_eb63a03f-7d69-4af8-81ac-0b80a5eb9adf/kube-rbac-proxy-metrics/0.log"
Jul 29 19:01:45.592694 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.592666    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"]
Jul 29 19:01:45.593080 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593055    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56ce7788-4842-491e-ad18-43c688114c24" containerName="payload-pre-processing"
Jul 29 19:01:45.593080 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593067    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="56ce7788-4842-491e-ad18-43c688114c24" containerName="payload-pre-processing"
Jul 29 19:01:45.593080 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593078    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy"
Jul 29 19:01:45.593080 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593083    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593090    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593096    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593109    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593114    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593130    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593136    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593146    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593152    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593158    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593163    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593171    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593176    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593182    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593187    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593247    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593254    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="56ce7788-4842-491e-ad18-43c688114c24" containerName="payload-pre-processing"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593260    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593267    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e7fd0a-3a51-4e96-8358-708e100cd921" containerName="istio-proxy"
Jul 29 19:01:45.593264 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593273    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerName="istio-proxy"
Jul 29 19:01:45.593913 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593279    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db3b54d1-f91e-4c5b-90e6-c5b524807dc9" containerName="istio-proxy"
Jul 29 19:01:45.593913 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593285    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e48a3423-4778-4fb5-9384-2ac7dc90a48d" containerName="istio-proxy"
Jul 29 19:01:45.593913 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593290    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy"
Jul 29 19:01:45.593913 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.593295    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="542f7807-2aa1-4ff7-8f74-7f052c8c3ab1" containerName="istio-proxy"
Jul 29 19:01:45.599248 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.599231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.602406 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.602236    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5k8jh\"/\"kube-root-ca.crt\""
Jul 29 19:01:45.602406 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.602237    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5k8jh\"/\"default-dockercfg-9zp9k\""
Jul 29 19:01:45.602406 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.602338    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5k8jh\"/\"openshift-service-ca.crt\""
Jul 29 19:01:45.604321 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.604297    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"]
Jul 29 19:01:45.669613 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.669587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-proc\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.669714 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.669631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-podres\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.669714 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.669665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-sys\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.669714 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.669705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-lib-modules\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.669847 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.669736    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k47hj\" (UniqueName: \"kubernetes.io/projected/d57d8338-7597-434c-8758-ccab785b3d72-kube-api-access-k47hj\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.770418 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.770391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-proc\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.770512 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.770435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-podres\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.770512 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.770465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-sys\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.770512 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.770485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-lib-modules\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.770512 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.770505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k47hj\" (UniqueName: \"kubernetes.io/projected/d57d8338-7597-434c-8758-ccab785b3d72-kube-api-access-k47hj\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.770652 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.770515    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-proc\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.770652 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.770572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-sys\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.770652 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.770616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-podres\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.770652 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.770637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d57d8338-7597-434c-8758-ccab785b3d72-lib-modules\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.777971 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.777934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k47hj\" (UniqueName: \"kubernetes.io/projected/d57d8338-7597-434c-8758-ccab785b3d72-kube-api-access-k47hj\") pod \"perf-node-gather-daemonset-75m6l\" (UID: \"d57d8338-7597-434c-8758-ccab785b3d72\") " pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:45.911464 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:45.911404    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:46.041835 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:46.041809    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"]
Jul 29 19:01:46.043903 ip-10-0-129-228 kubenswrapper[2580]: W0729 19:01:46.043871    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd57d8338_7597_434c_8758_ccab785b3d72.slice/crio-f9ebca6a3be25ad88a2c57504ce19dfeb6daaea83528003a45bc9718cdd461d8 WatchSource:0}: Error finding container f9ebca6a3be25ad88a2c57504ce19dfeb6daaea83528003a45bc9718cdd461d8: Status 404 returned error can't find the container with id f9ebca6a3be25ad88a2c57504ce19dfeb6daaea83528003a45bc9718cdd461d8
Jul 29 19:01:46.187520 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:46.187454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l" event={"ID":"d57d8338-7597-434c-8758-ccab785b3d72","Type":"ContainerStarted","Data":"bfe363942707b08efe8e65558e45dc65c0ab31946b9a8ac33b6c5db3587fd4a0"}
Jul 29 19:01:46.187520 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:46.187512    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l" event={"ID":"d57d8338-7597-434c-8758-ccab785b3d72","Type":"ContainerStarted","Data":"f9ebca6a3be25ad88a2c57504ce19dfeb6daaea83528003a45bc9718cdd461d8"}
Jul 29 19:01:46.187695 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:46.187567    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:46.202438 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:46.202398    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l" podStartSLOduration=1.202385881 podStartE2EDuration="1.202385881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:01:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:01:46.200624009 +0000 UTC m=+4157.470122803" watchObservedRunningTime="2026-07-29 19:01:46.202385881 +0000 UTC m=+4157.471884663"
Jul 29 19:01:46.658878 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:46.658846    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-tdn25_0d7c5d66-1e6d-46d2-b831-522a4a58b3eb/download-server/0.log"
Jul 29 19:01:48.386873 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:48.386844    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v8gbz_bedfbcbc-a007-4186-9405-989cdc0afd42/dns/0.log"
Jul 29 19:01:48.405826 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:48.405802    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v8gbz_bedfbcbc-a007-4186-9405-989cdc0afd42/kube-rbac-proxy/0.log"
Jul 29 19:01:48.453973 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:48.453933    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7b96n_f9337a15-2a68-4c41-82c3-6cf81a81e1f6/dns-node-resolver/0.log"
Jul 29 19:01:49.069218 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:49.069190    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zh4v2_5b7f4a89-2914-4a7c-a5cd-69a9ab162439/node-ca/0.log"
Jul 29 19:01:51.346830 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:51.346802    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8mqfn_066e69d4-22ad-4231-b7c9-4b55475efeee/serve-healthcheck-canary/0.log"
Jul 29 19:01:51.888828 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:51.888801    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9nhcg_5e5a0c7c-c535-4652-968b-f177289a242e/kube-rbac-proxy/0.log"
Jul 29 19:01:51.908569 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:51.908546    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9nhcg_5e5a0c7c-c535-4652-968b-f177289a242e/exporter/0.log"
Jul 29 19:01:51.929420 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:51.929384    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9nhcg_5e5a0c7c-c535-4652-968b-f177289a242e/extractor/0.log"
Jul 29 19:01:52.201613 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:52.201548    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5k8jh/perf-node-gather-daemonset-75m6l"
Jul 29 19:01:54.462713 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:54.462681    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jkjr6_a71e79b4-cd7a-4d57-947e-fdbd388cbd60/manager/0.log"
Jul 29 19:01:54.859081 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:54.859044    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xldbf_2897d9aa-2f97-46da-b23f-d7cc6cb28589/manager/0.log"
Jul 29 19:01:55.326498 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:55.326465    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-d4fkp_620f475b-5a57-46e4-abe0-265834f558ac/manager/0.log"
Jul 29 19:01:55.343280 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:55.343260    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d84c6c79c-d4fkp_620f475b-5a57-46e4-abe0-265834f558ac/copy-manifests/0.log"
Jul 29 19:01:56.806623 ip-10-0-129-228 kubenswrapper[2580]: I0729 19:01:56.806585    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7ff9fc5cb5-c6hw2_be34e128-7f01-4b11-93c7-ce5197de6019/manager/0.log"