Jul 29 19:17:12.654378 ip-10-0-129-107 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 19:17:13.071478 ip-10-0-129-107 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 19:17:13.071478 ip-10-0-129-107 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 19:17:13.071478 ip-10-0-129-107 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 19:17:13.071478 ip-10-0-129-107 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 19:17:13.071478 ip-10-0-129-107 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 19:17:13.072171 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.071560    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 19:17:13.076197 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076181    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:17:13.076197 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076196    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076200    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076204    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076206    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076210    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076212    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076216    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076219    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076222    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076225    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076228    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076230    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076233    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076235    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076238    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076241    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076243    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076246    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076249    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076251    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:17:13.076263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076254    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076258    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076262    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076265    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076274    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076277    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076280    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076283    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076285    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076288    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076290    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076293    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076296    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076298    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076301    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076304    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076306    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076310    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:17:13.076733 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076313    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076316    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076319    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076322    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076324    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076327    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076330    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076332    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076335    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076337    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076340    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076342    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076346    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076349    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076351    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076354    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076357    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076360    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076363    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076366    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076369    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:17:13.077177 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076371    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076374    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076377    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076379    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076382    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076385    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076388    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076391    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076393    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076396    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076398    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076401    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076404    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076406    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076409    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076411    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076414    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076416    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076419    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076422    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:17:13.077710 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076424    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076427    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076429    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076432    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076434    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076437    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076439    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076444    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076446    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076449    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076451    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076455    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076458    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076461    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076464    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076467    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076470    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076473    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.076475    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:17:13.078329 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077171    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077178    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077182    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077184    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077187    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077190    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077193    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077198    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077201    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077205    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077208    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077211    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077214    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077216    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077219    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077222    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077225    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077227    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077230    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:17:13.078788 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077232    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077236    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077239    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077242    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077244    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077247    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077249    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077252    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077255    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077257    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077260    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077263    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077265    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077268    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077271    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077275    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077278    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077280    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077283    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077286    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:17:13.079275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077289    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077291    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077294    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077297    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077299    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077302    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077304    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077307    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077309    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077312    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077315    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077317    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077320    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077322    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077325    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077328    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077330    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077333    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077335    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077338    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:17:13.079758 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077340    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077343    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077346    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077348    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077351    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077353    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077358    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077361    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077364    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077367    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077370    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077372    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077375    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077377    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077380    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077382    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077386    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077388    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077391    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077393    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:17:13.080255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077396    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077399    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077401    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077404    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077406    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077410    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077413    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077415    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077418    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077420    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077423    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077425    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077428    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077431    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077433    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077436    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077438    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077441    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077443    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.077446    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:17:13.080748 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078462    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078469    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078473    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078476    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078479    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078482    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078484    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078489    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078493    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078496    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078499    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078502    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078505    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078507    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078510    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078512    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078515    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078518    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078520    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078523    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:17:13.081252 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078525    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078528    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078530    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078533    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078535    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078538    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078540    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078543    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078545    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078548    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078550    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078553    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078556    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078558    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078561    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078563    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078565    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078568    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078571    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078573    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:17:13.081737 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078576    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078579    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078581    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078584    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078587    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078589    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078592    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078594    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078597    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078600    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078602    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078605    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078608    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078611    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078613    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078616    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078618    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078621    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078623    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:17:13.082263 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078626    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078628    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078631    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078633    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078636    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078639    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078641    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078644    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078647    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078650    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078653    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078656    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078659    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078662    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078665    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078669    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078672    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078675    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078677    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:17:13.082765 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078680    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078682    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078687    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078689    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078692    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078694    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078697    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078700    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078702    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078704    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078707    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078709    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078711    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078714    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078717    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078719    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078722    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078725    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078727    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078730    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:17:13.083244 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.078732    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078802    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078811    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078819    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078825    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078832    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078837    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078843    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078849    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078855    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078860    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078864    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078867    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078870    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078874    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078877    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078880    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078882    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078885    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078891    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078894    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078897    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078900    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078903    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 19:17:13.083734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078907    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078911    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078914    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078925    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078928    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078932    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078935    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078938    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078941    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078945    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078948    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078955    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078958    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078961    2566 flags.go:64] FLAG: --enable-server="true"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078964    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078968    2566 flags.go:64] FLAG: --event-burst="100"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078971    2566 flags.go:64] FLAG: --event-qps="50"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078974    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078978    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078981    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078984    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078987    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078993    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078996    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.078999    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 19:17:13.084313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079002    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079005    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079008    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079011    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079014    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079016    2566 flags.go:64] FLAG: --feature-gates=""
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079020    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079023    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079026    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079029    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079032    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079035    2566 flags.go:64] FLAG: --help="false"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079040    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-129-107.ec2.internal"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079043    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079046    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079048    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079052    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079055    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079059    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079062    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079065    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079067    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079070    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079073    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 19:17:13.084937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079076    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079079    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079082    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079097    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079100    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079104    2566 flags.go:64] FLAG: --lock-file=""
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079107    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079110    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079113    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079118    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079121    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079124    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079127    2566 flags.go:64] FLAG: --logging-format="text"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079130    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079134    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079137    2566 flags.go:64] FLAG: --manifest-url=""
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079139    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079149    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079152    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079156    2566 flags.go:64] FLAG: --max-pods="110"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079161    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079164    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079167    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079170    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079173    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 19:17:13.085563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079176    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079179    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079187    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079190    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079194    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079196    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079199    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079205    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079207    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079210    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079214    2566 flags.go:64] FLAG: --port="10250"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079217    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079220    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-000ee0c6f75fd5e9e"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079224    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079227    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079230    2566 flags.go:64] FLAG: --register-node="true"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079233    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079237    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079240    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079242    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079245    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079248    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079251    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079254    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079257    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 19:17:13.086219 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079260    2566 flags.go:64] FLAG: --runonce="false"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079262    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079267    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079270    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079272    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079276    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079279    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079282    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079285    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079288    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079292    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079294    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079297    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079300    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079303    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079306    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079312    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079314    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079317    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079322    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079326    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079329    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079332    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079335    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079337    2566 flags.go:64] FLAG: --v="2"
Jul 29 19:17:13.086833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079341    2566 flags.go:64] FLAG: --version="false"
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079345    2566 flags.go:64] FLAG: --vmodule=""
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079348    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079351    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079467    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079470    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079474    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079477    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079479    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079483    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079485    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079489    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079491    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079494    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079497    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079499    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079502    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079504    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079507    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079509    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:17:13.087463 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079512    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079514    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079517    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079520    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079522    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079525    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079528    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079533    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079536    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079539    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079541    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079544    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079546    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079549    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079552    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079554    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079558    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079561    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079564    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:17:13.087952 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079568    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079572    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079574    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079577    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079580    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079583    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079586    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079588    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079591    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079594    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079596    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079599    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079601    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079604    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079606    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079609    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079612    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079614    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079617    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079619    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:17:13.088422 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079623    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079626    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079628    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079631    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079634    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079636    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079639    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079641    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079644    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079647    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079649    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079652    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079654    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079657    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079659    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079662    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079664    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079667    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079669    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:17:13.088898 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079672    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079676    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079679    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079682    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079685    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079687    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079690    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079693    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079696    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079698    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079701    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079703    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079706    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079710    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079713    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079716    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079718    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079721    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079724    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:17:13.089448 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079726    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079728    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079731    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079734    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079736    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.079738    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.079747    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.086300    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.086315    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.086331    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.086340    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086394    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086398    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086402    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:17:13.089896 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086405    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086408    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086411    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086414    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086416    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086419    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086422    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086425    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086427    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086429    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086432    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086435    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086437    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086440    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086442    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086445    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086448    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086451    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086453    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086456    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:17:13.090255 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086458    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086461    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086463    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086466    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086469    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086471    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086475    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086478    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086480    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086483    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086486    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086490    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086493    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086496    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086499    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086502    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086505    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086507    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086510    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:17:13.090743 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086512    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086515    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086518    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086520    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086523    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086526    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086528    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086531    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086534    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086536    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086539    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086541    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086544    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086546    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086549    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086551    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086554    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086556    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086560    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086563    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:17:13.091222 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086566    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086569    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086571    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086574    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086577    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086579    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086582    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086585    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086587    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086590    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086594    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086597    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086599    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086602    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086605    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086608    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086611    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086613    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086616    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:17:13.091738 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086619    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086621    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086624    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086626    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086629    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086631    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086634    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086636    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086640    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086643    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086645    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086649    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086652    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086656    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086659    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086662    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086664    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:17:13.092206 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086667    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.086672    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086803    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086809    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086813    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086816    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086819    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086822    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086824    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086827    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086830    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086832    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086835    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086837    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:17:13.092620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086840    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086843    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086845    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086848    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086851    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086854    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086856    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086859    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086861    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086864    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086867    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086869    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086872    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086875    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086877    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086879    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086882    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086885    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086889    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086892    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:17:13.092955 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086895    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086898    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086901    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086903    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086906    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086909    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086912    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086914    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086917    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086920    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086922    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086925    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086927    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086929    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086932    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086934    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086937    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086939    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086941    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086944    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:17:13.093442 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086946    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086949    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086951    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086954    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086957    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086960    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086963    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086965    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086967    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086970    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086973    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086975    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086978    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086980    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086982    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086986    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086988    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086990    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086993    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086995    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:17:13.093914 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.086998    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087000    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087002    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087005    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087007    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087010    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087012    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087015    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087017    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087020    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087022    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087025    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087027    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087029    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087032    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087036    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087040    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087043    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087045    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:17:13.094424 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087047    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087050    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087052    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087055    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087058    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087060    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087062    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:13.087065    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.087070    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.087177    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.089133    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.091831    2566 server.go:1019] "Starting client certificate rotation"
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.091934    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 19:17:13.094930 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.091963    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 19:17:13.097464 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.097441    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 19:17:13.098672 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.098652    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 19:17:13.113210 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.113188    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 29 19:17:13.115294 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.115279    2566 log.go:25] "Validated CRI v1 image API"
Jul 29 19:17:13.117281 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.117218    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 19:17:13.119620 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.119597    2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bc16be07-421e-44a1-85aa-224c072854e3:/dev/nvme0n1p3 f8a4abed-8cf5-49a5-8a0c-1091597289b9:/dev/nvme0n1p4]
Jul 29 19:17:13.119672 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.119620    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 19:17:13.122876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.122861    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 19:17:13.124835 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.124726    2566 manager.go:216] Machine: {Timestamp:2026-07-29 19:17:13.123398305 +0000 UTC m=+0.349521618 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099054 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec260fe745245ed0d990852d80921c24 SystemUUID:ec260fe7-4524-5ed0-d990-852d80921c24 BootID:72fa2817-a230-47bb-a0a3-da992d5b0946 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e5:81:25:08:e9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e5:81:25:08:e9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:fe:5d:c8:3a:e4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 19:17:13.124835 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.124831    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 19:17:13.124952 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.124940    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 19:17:13.127695 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.127673    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 19:17:13.127857 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.127698    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-107.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 19:17:13.128462 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.128452    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 19:17:13.128500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.128465    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 19:17:13.128500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.128472    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 19:17:13.129196 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.129186    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 19:17:13.130018 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.130009    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 19:17:13.131732 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.131722    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 19:17:13.131834 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.131825    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 19:17:13.131876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.131852    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 19:17:13.131876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.131861    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 19:17:13.131876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.131874    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 19:17:13.131981 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.131883    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 29 19:17:13.131981 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.131891    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 19:17:13.132231 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.132219    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 19:17:13.132286 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.132238    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 19:17:13.134554 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.134540    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 19:17:13.136305 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.136289    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 19:17:13.136382 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.136328    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 19:17:13.137954 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.137942    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 19:17:13.137954 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.137962    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 19:17:13.138074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.137972    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 19:17:13.138074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.137980    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 19:17:13.138074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.137993    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 19:17:13.138074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.138001    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 19:17:13.138074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.138013    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 19:17:13.138074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.138021    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 19:17:13.138074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.138030    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 19:17:13.138074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.138038    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 19:17:13.138074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.138051    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 19:17:13.138074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.138059    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 19:17:13.138926 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.138914    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 19:17:13.138981 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.138928    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 19:17:13.143120 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.143081    2566 server.go:1268] "Started kubelet"
Jul 29 19:17:13.143216 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.143180    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 19:17:13.143315 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.143247    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 19:17:13.143408 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.143329    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 19:17:13.143912 ip-10-0-129-107 systemd[1]: Started Kubernetes Kubelet.
Jul 29 19:17:13.145057 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.145043    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 19:17:13.145681 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.145664    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 19:17:13.148259 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.148237    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 19:17:13.151629 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.151610    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 19:17:13.151629 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.151623    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 19:17:13.151770 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.151740    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:13.152153 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152135    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 19:17:13.152153 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152152    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 19:17:13.152283 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152183    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 19:17:13.152283 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152260    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 19:17:13.152283 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152268    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 19:17:13.152509 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152490    2566 factory.go:55] Registering systemd factory
Jul 29 19:17:13.152599 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152529    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 29 19:17:13.152725 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152711    2566 factory.go:153] Registering CRI-O factory
Jul 29 19:17:13.152725 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152725    2566 factory.go:223] Registration of the crio container factory successfully
Jul 29 19:17:13.152840 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152776    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 19:17:13.152840 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152801    2566 factory.go:103] Registering Raw factory
Jul 29 19:17:13.152840 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.152816    2566 manager.go:1196] Started watching for new ooms in manager
Jul 29 19:17:13.153256 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.153241    2566 manager.go:349] Starting recovery of all containers
Jul 29 19:17:13.165123 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.165051    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 19:17:13.165249 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.165220    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-107.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 19:17:13.166308 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.166277    2566 manager.go:354] Recovery completed
Jul 29 19:17:13.166825 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.166796    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-107.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 19:17:13.166901 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.165744    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-107.ec2.internal.18c6d8c2b94cd326  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-107.ec2.internal,UID:ip-10-0-129-107.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-107.ec2.internal,},FirstTimestamp:2026-07-29 19:17:13.143055142 +0000 UTC m=+0.369178434,LastTimestamp:2026-07-29 19:17:13.143055142 +0000 UTC m=+0.369178434,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-107.ec2.internal,}"
Jul 29 19:17:13.167022 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.167000    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 19:17:13.167149 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.167134    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-107.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 19:17:13.171929 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.171916    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:17:13.174903 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.174872    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:17:13.174976 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.174921    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:17:13.174976 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.174937    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:17:13.175535 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.175522    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 19:17:13.175586 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.175536    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 19:17:13.175586 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.175554    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 19:17:13.175586 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.175559    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:17:13.176290 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.176275    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:17:13.176339 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.176300    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:17:13.176339 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.176310    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:17:13.176911 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.176848    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-107.ec2.internal.18c6d8c2bb32c748  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-107.ec2.internal,UID:ip-10-0-129-107.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-107.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-107.ec2.internal,},FirstTimestamp:2026-07-29 19:17:13.1749026 +0000 UTC m=+0.401025897,LastTimestamp:2026-07-29 19:17:13.1749026 +0000 UTC m=+0.401025897,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-107.ec2.internal,}"
Jul 29 19:17:13.177141 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.177130    2566 policy_none.go:49] "None policy: Start"
Jul 29 19:17:13.177185 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.177145    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 19:17:13.177185 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.177156    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 19:17:13.178083 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.178073    2566 policy_none.go:47] "Start"
Jul 29 19:17:13.179205 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.179187    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hd5lt"
Jul 29 19:17:13.187099 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.187021    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-107.ec2.internal.18c6d8c2bb333204  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-107.ec2.internal,UID:ip-10-0-129-107.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-107.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-107.ec2.internal,},FirstTimestamp:2026-07-29 19:17:13.174929924 +0000 UTC m=+0.401053216,LastTimestamp:2026-07-29 19:17:13.174929924 +0000 UTC m=+0.401053216,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-107.ec2.internal,}"
Jul 29 19:17:13.197515 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.197496    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hd5lt"
Jul 29 19:17:13.203040 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.202968    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-107.ec2.internal.18c6d8c2bb3365dc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-107.ec2.internal,UID:ip-10-0-129-107.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-107.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-107.ec2.internal,},FirstTimestamp:2026-07-29 19:17:13.174943196 +0000 UTC m=+0.401066491,LastTimestamp:2026-07-29 19:17:13.174943196 +0000 UTC m=+0.401066491,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-107.ec2.internal,}"
Jul 29 19:17:13.216828 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.216669    2566 manager.go:337] "Starting Device Plugin manager"
Jul 29 19:17:13.216929 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.216866    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 19:17:13.216929 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.216878    2566 server.go:87] "Starting device plugin registration server"
Jul 29 19:17:13.217105 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.217081    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 19:17:13.217156 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.217108    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 19:17:13.217223 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.217196    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 19:17:13.217366 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.217340    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 19:17:13.217366 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:13.217353    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 19:17:13.217878 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.217858    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 19:17:13.217939 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:13.217913    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:15.047639 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.047599    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:17:15.048688 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.048535    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:17:15.048688 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.048566    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:17:15.048688 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.048582    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:17:15.048688 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.048610    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.061902 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.061853    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.061902 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.061876    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-107.ec2.internal\": node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:15.079317 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.079293    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:15.094254 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.094215    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 19:17:15.095334 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.095318    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 19:17:15.095397 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.095357    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 19:17:15.095397 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.095381    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 19:17:15.095482 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.095421    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 19:17:15.098265 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.098247    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 19:17:15.179774 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.179725    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:15.196154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.196125    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-107.ec2.internal"]
Jul 29 19:17:15.196232 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.196222    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:17:15.197766 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.197751    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:17:15.197841 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.197781    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:17:15.197841 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.197790    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:17:15.198943 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.198932    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:17:15.199047 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.199034    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.199082 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.199072    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:17:15.199662 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.199641    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:17:15.199756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.199672    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:17:15.199756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.199685    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:17:15.199756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.199693    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:17:15.199756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.199712    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:17:15.199756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.199721    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:17:15.200814 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.200801    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.200859 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.200824    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:17:15.201410 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.201394    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:17:15.201484 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.201423    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:17:15.201484 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.201443    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:17:15.225068 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.225042    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-107.ec2.internal\" not found" node="ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.229535 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.229520    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-107.ec2.internal\" not found" node="ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.280131 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.280105    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:15.285977 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.285955    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2e8713941fec20c0804fb302fd41b7b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal\" (UID: \"d2e8713941fec20c0804fb302fd41b7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.286044 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.285980    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2e8713941fec20c0804fb302fd41b7b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal\" (UID: \"d2e8713941fec20c0804fb302fd41b7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.286044 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.285997    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ccf7f81c6540eccb14cc3f753f1ba29f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-107.ec2.internal\" (UID: \"ccf7f81c6540eccb14cc3f753f1ba29f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.380251 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.380175    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:15.386577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.386556    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ccf7f81c6540eccb14cc3f753f1ba29f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-107.ec2.internal\" (UID: \"ccf7f81c6540eccb14cc3f753f1ba29f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.386640 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.386582    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2e8713941fec20c0804fb302fd41b7b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal\" (UID: \"d2e8713941fec20c0804fb302fd41b7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.386640 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.386599    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2e8713941fec20c0804fb302fd41b7b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal\" (UID: \"d2e8713941fec20c0804fb302fd41b7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.386640 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.386624    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2e8713941fec20c0804fb302fd41b7b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal\" (UID: \"d2e8713941fec20c0804fb302fd41b7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.386740 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.386642    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2e8713941fec20c0804fb302fd41b7b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal\" (UID: \"d2e8713941fec20c0804fb302fd41b7b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.386740 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.386663    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ccf7f81c6540eccb14cc3f753f1ba29f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-107.ec2.internal\" (UID: \"ccf7f81c6540eccb14cc3f753f1ba29f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.480760 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.480716    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:15.529231 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.529201    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.533134 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.532913    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:15.581019 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.580992    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:15.681531 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.681467    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:15.781978 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.781948    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:15.820379 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.820350    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 19:17:15.820771 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.820529    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 19:17:15.883022 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.882993    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:15.933431 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.933363    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 19:12:14 +0000 UTC" deadline="2028-02-23 14:01:59.105868689 +0000 UTC"
Jul 29 19:17:15.933431 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:15.933399    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13770h44m43.172472672s"
Jul 29 19:17:15.983756 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:15.983722    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:16.077074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.077048    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:17:16.078011 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.077996    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:17:16.078073 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.078026    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:17:16.078073 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.078036    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:17:16.084645 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:16.084625    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:16.153036 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.153014    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 19:17:16.185393 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:16.185332    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-107.ec2.internal\" not found"
Jul 29 19:17:16.225863 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.225841    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 19:17:16.277938 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:16.277907    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e8713941fec20c0804fb302fd41b7b.slice/crio-986c8b93449b17cd2a0f16b856b06c13e6bb293c2707204bca326781a9e2dc7f WatchSource:0}: Error finding container 986c8b93449b17cd2a0f16b856b06c13e6bb293c2707204bca326781a9e2dc7f: Status 404 returned error can't find the container with id 986c8b93449b17cd2a0f16b856b06c13e6bb293c2707204bca326781a9e2dc7f
Jul 29 19:17:16.280714 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:16.280694    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccf7f81c6540eccb14cc3f753f1ba29f.slice/crio-7a9cd1ffe9beb74789afc689bc8013cedcee87da9728405d933d05d9dba85716 WatchSource:0}: Error finding container 7a9cd1ffe9beb74789afc689bc8013cedcee87da9728405d933d05d9dba85716: Status 404 returned error can't find the container with id 7a9cd1ffe9beb74789afc689bc8013cedcee87da9728405d933d05d9dba85716
Jul 29 19:17:16.282844 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.282831    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:16.285794 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.285775    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 19:17:16.293582 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.293565    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 19:17:16.294569 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.294555    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal"
Jul 29 19:17:16.301522 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.301508    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 19:17:16.305835 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.305819    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 19:17:16.863134 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.863102    2566 apiserver.go:52] "Watching apiserver"
Jul 29 19:17:16.871866 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.871659    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 19:17:16.874871 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.874839    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-99b48","openshift-multus/multus-9rhjr","openshift-network-operator/iptables-alerter-x8qd9","openshift-ovn-kubernetes/ovnkube-node-ckmtk","kube-system/global-pull-secret-syncer-ppw48","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8","openshift-cluster-node-tuning-operator/tuned-ccqx5","openshift-image-registry/node-ca-86g76","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal","openshift-multus/multus-additional-cni-plugins-gczs9","openshift-multus/network-metrics-daemon-cmpgk","openshift-network-diagnostics/network-check-target-m7mvb","kube-system/konnectivity-agent-wl5d4","kube-system/kube-apiserver-proxy-ip-10-0-129-107.ec2.internal"]
Jul 29 19:17:16.876810 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.876784    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.878692 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.878675    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:16.881235 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.881215    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 19:17:16.881474 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.881442    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:17:16.881689 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.881674    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gvfqr\""
Jul 29 19:17:16.884506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.884487    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.884702 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.884680    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:16.884776 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.884707    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 19:17:16.884776 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:16.884762    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:16.884876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.884772    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-j9lch\""
Jul 29 19:17:16.884924 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.884893    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x8qd9"
Jul 29 19:17:16.884973 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.884945    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-86g76"
Jul 29 19:17:16.885430 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.885037    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 19:17:16.885430 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.885142    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:16.885430 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.885351    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 19:17:16.886435 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.886024    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-99b48"
Jul 29 19:17:16.887881 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.887640    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:16.888422 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.888405    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 19:17:16.889195 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.888614    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 19:17:16.889195 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.888784    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 19:17:16.889195 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.888959    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 19:17:16.889195 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.889183    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:16.890059 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.890038    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 19:17:16.890279 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.890259    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 19:17:16.890503 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.890486    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.891635    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:16.891719    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.891739    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:16.891817    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.893953    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea48fa24-04c4-434f-9cb9-eb941d6fff89-kubelet-config\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.893996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a463d6f0-c349-4a93-8d64-ca9fe04cdc46-konnectivity-ca\") pod \"konnectivity-agent-wl5d4\" (UID: \"a463d6f0-c349-4a93-8d64-ca9fe04cdc46\") " pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894025    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46ms5\" (UniqueName: \"kubernetes.io/projected/ee73b778-04f0-4c42-8769-1fbfbdbab0ab-kube-api-access-46ms5\") pod \"iptables-alerter-x8qd9\" (UID: \"ee73b778-04f0-4c42-8769-1fbfbdbab0ab\") " pod="openshift-network-operator/iptables-alerter-x8qd9"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894058    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-cni-bin\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894168    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-sysctl-d\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894197    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-lib-modules\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894227    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894271    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-run-netns\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894398    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-run-openvswitch\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894430    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-log-socket\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894518    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zp5f\" (UniqueName: \"kubernetes.io/projected/b2377041-6074-4084-9042-655f433c91ed-kube-api-access-8zp5f\") pod \"node-resolver-99b48\" (UID: \"b2377041-6074-4084-9042-655f433c91ed\") " pod="openshift-dns/node-resolver-99b48"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894574    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a463d6f0-c349-4a93-8d64-ca9fe04cdc46-agent-certs\") pod \"konnectivity-agent-wl5d4\" (UID: \"a463d6f0-c349-4a93-8d64-ca9fe04cdc46\") " pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:16.895472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894598    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894754    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-var-lib-openvswitch\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-sys-fs\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894897    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea48fa24-04c4-434f-9cb9-eb941d6fff89-dbus\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.894926    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2377041-6074-4084-9042-655f433c91ed-hosts-file\") pod \"node-resolver-99b48\" (UID: \"b2377041-6074-4084-9042-655f433c91ed\") " pod="openshift-dns/node-resolver-99b48"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.895049    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-slash\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.895076    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-etc-openvswitch\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.895106    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.895135    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d748fef7-568c-4f8c-9698-6f93e239f253-serviceca\") pod \"node-ca-86g76\" (UID: \"d748fef7-568c-4f8c-9698-6f93e239f253\") " pod="openshift-image-registry/node-ca-86g76"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.895164    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-tuned\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.895360    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8sq9\" (UniqueName: \"kubernetes.io/projected/49adb559-ddd6-476d-b01d-42e50ca37a83-kube-api-access-x8sq9\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.895394    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee73b778-04f0-4c42-8769-1fbfbdbab0ab-iptables-alerter-script\") pod \"iptables-alerter-x8qd9\" (UID: \"ee73b778-04f0-4c42-8769-1fbfbdbab0ab\") " pod="openshift-network-operator/iptables-alerter-x8qd9"
Jul 29 19:17:16.896253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.895418    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-sysctl-conf\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898007    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/82f6bd75-5e49-43df-8d20-b008e485353b-ovn-node-metrics-cert\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898063    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-modprobe-d\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898095    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-kubernetes\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/719397e8-732b-4941-bfc3-b1dc71d2371a-tmp\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898152    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qxmv\" (UniqueName: \"kubernetes.io/projected/d748fef7-568c-4f8c-9698-6f93e239f253-kube-api-access-5qxmv\") pod \"node-ca-86g76\" (UID: \"d748fef7-568c-4f8c-9698-6f93e239f253\") " pod="openshift-image-registry/node-ca-86g76"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898191    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d748fef7-568c-4f8c-9698-6f93e239f253-host\") pod \"node-ca-86g76\" (UID: \"d748fef7-568c-4f8c-9698-6f93e239f253\") " pod="openshift-image-registry/node-ca-86g76"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898217    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-sysconfig\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898278    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw6tw\" (UniqueName: \"kubernetes.io/projected/719397e8-732b-4941-bfc3-b1dc71d2371a-kube-api-access-lw6tw\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898314    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-kubelet\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898350    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-sys\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898383    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-host\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898413    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-node-log\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898443    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-socket-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898481    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-device-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898521    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-systemd-units\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.898707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898561    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/82f6bd75-5e49-43df-8d20-b008e485353b-env-overrides\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898599    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8246\" (UniqueName: \"kubernetes.io/projected/82f6bd75-5e49-43df-8d20-b008e485353b-kube-api-access-v8246\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898658    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-run\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898690    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-run-ovn\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898729    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-run-ovn-kubernetes\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898761    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-cni-netd\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898879    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-var-lib-kubelet\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898932    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2377041-6074-4084-9042-655f433c91ed-tmp-dir\") pod \"node-resolver-99b48\" (UID: \"b2377041-6074-4084-9042-655f433c91ed\") " pod="openshift-dns/node-resolver-99b48"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.898984    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/82f6bd75-5e49-43df-8d20-b008e485353b-ovnkube-script-lib\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.899106    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-registration-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.899203    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.899223    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-systemd\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.899241    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee73b778-04f0-4c42-8769-1fbfbdbab0ab-host-slash\") pod \"iptables-alerter-x8qd9\" (UID: \"ee73b778-04f0-4c42-8769-1fbfbdbab0ab\") " pod="openshift-network-operator/iptables-alerter-x8qd9"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.899331    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-run-systemd\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.899557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.899364    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/82f6bd75-5e49-43df-8d20-b008e485353b-ovnkube-config\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.900199 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.899914    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nlf74\""
Jul 29 19:17:16.900252 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.900204    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 19:17:16.901564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.900421    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 19:17:16.901564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.900493    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 19:17:16.901564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.900753    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 19:17:16.901564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.900901    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 19:17:16.901564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.900922    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 19:17:16.901564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.901084    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nbnlk\""
Jul 29 19:17:16.901564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.901337    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 19:17:16.901959 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.901646    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 19:17:16.902089 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.902014    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mnnpm\""
Jul 29 19:17:16.903564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.902222    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 19:17:16.903564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.902239    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 19:17:16.903564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.902406    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 19:17:16.903564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.902518    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 19:17:16.903564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.902679    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g6hf5\""
Jul 29 19:17:16.903564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.902797    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:17:16.903564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.902837    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 19:17:16.905375 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.904622    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 19:17:16.905375 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.905083    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 19:17:16.905375 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.905230    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4c5xf\""
Jul 29 19:17:16.905375 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.905262    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dnhfq\""
Jul 29 19:17:16.906538 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.906519    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-t2h9c\""
Jul 29 19:17:16.911830 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.911809    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 19:17:16.922031 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.922010    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-h5j27"
Jul 29 19:17:16.941024 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.940981    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-h5j27"
Jul 29 19:17:16.984010 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.983985    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 19:17:16.999583 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999561    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-sys\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.999726 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999590    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-host\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:16.999726 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999615    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-node-log\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.999726 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999639    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-socket-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:16.999726 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-device-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:16.999726 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999700    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-systemd-units\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:16.999726 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999723    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/82f6bd75-5e49-43df-8d20-b008e485353b-env-overrides\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999747    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8246\" (UniqueName: \"kubernetes.io/projected/82f6bd75-5e49-43df-8d20-b008e485353b-kube-api-access-v8246\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999776    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-socket-dir-parent\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999804    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-var-lib-cni-multus\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999827    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-run\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999850    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-run-ovn\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999865    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-device-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999875    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-run-ovn-kubernetes\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999899    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-cni-netd\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999907    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-sys\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999923    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qfk9\" (UniqueName: \"kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9\") pod \"network-check-target-m7mvb\" (UID: \"2db48639-ee08-485b-874c-c64954b418dc\") " pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999937    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-host\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999950    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-var-lib-cni-bin\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999961    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-node-log\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:16.999975    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-daemon-config\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.000022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000001    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-var-lib-kubelet\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2377041-6074-4084-9042-655f433c91ed-tmp-dir\") pod \"node-resolver-99b48\" (UID: \"b2377041-6074-4084-9042-655f433c91ed\") " pod="openshift-dns/node-resolver-99b48"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000066    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/82f6bd75-5e49-43df-8d20-b008e485353b-ovnkube-script-lib\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000065    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-socket-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000091    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-registration-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000152    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-run-ovn-kubernetes\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000160    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-run-ovn\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000174    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-systemd-units\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000175    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-registration-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000191    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000193    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-cni-netd\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000240    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-cnibin\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000278    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-var-lib-kubelet\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000288    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-systemd\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000305    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-etc-selinux\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000315    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000378    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-systemd\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.000749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000410    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee73b778-04f0-4c42-8769-1fbfbdbab0ab-host-slash\") pod \"iptables-alerter-x8qd9\" (UID: \"ee73b778-04f0-4c42-8769-1fbfbdbab0ab\") " pod="openshift-network-operator/iptables-alerter-x8qd9"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-run-systemd\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000476    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-run\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/82f6bd75-5e49-43df-8d20-b008e485353b-ovnkube-config\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000511    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-run-systemd\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000516    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-system-cni-dir\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000541    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-run-k8s-cni-cncf-io\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000476    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee73b778-04f0-4c42-8769-1fbfbdbab0ab-host-slash\") pod \"iptables-alerter-x8qd9\" (UID: \"ee73b778-04f0-4c42-8769-1fbfbdbab0ab\") " pod="openshift-network-operator/iptables-alerter-x8qd9"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000564    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2377041-6074-4084-9042-655f433c91ed-tmp-dir\") pod \"node-resolver-99b48\" (UID: \"b2377041-6074-4084-9042-655f433c91ed\") " pod="openshift-dns/node-resolver-99b48"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea48fa24-04c4-434f-9cb9-eb941d6fff89-kubelet-config\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000626    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea48fa24-04c4-434f-9cb9-eb941d6fff89-kubelet-config\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000647    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a463d6f0-c349-4a93-8d64-ca9fe04cdc46-konnectivity-ca\") pod \"konnectivity-agent-wl5d4\" (UID: \"a463d6f0-c349-4a93-8d64-ca9fe04cdc46\") " pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000656    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/82f6bd75-5e49-43df-8d20-b008e485353b-env-overrides\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000681    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46ms5\" (UniqueName: \"kubernetes.io/projected/ee73b778-04f0-4c42-8769-1fbfbdbab0ab-kube-api-access-46ms5\") pod \"iptables-alerter-x8qd9\" (UID: \"ee73b778-04f0-4c42-8769-1fbfbdbab0ab\") " pod="openshift-network-operator/iptables-alerter-x8qd9"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000723    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-cni-bin\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000753    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-cni-dir\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000777    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-sysctl-d\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.001539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000800    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-lib-modules\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000824    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000833    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-cni-bin\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000845    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/82f6bd75-5e49-43df-8d20-b008e485353b-ovnkube-script-lib\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000848    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-run-netns\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000871    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-run-openvswitch\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000886    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-log-socket\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000901    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a5e3afa4-7206-4362-9c74-32b04f05d4d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000919    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zp5f\" (UniqueName: \"kubernetes.io/projected/b2377041-6074-4084-9042-655f433c91ed-kube-api-access-8zp5f\") pod \"node-resolver-99b48\" (UID: \"b2377041-6074-4084-9042-655f433c91ed\") " pod="openshift-dns/node-resolver-99b48"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000934    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a463d6f0-c349-4a93-8d64-ca9fe04cdc46-agent-certs\") pod \"konnectivity-agent-wl5d4\" (UID: \"a463d6f0-c349-4a93-8d64-ca9fe04cdc46\") " pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000953    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000959    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-sysctl-d\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000971    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dcfa88a-bacc-40bc-bb01-fb73b300931d-cni-binary-copy\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000973    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-run-netns\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.000983    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-lib-modules\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.000933    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001001    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-hostroot\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001033    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-run-openvswitch\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.002322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001043    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001052    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-conf-dir\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.001082    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret podName:ea48fa24-04c4-434f-9cb9-eb941d6fff89 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:17.501054341 +0000 UTC m=+2.996815706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret") pod "global-pull-secret-syncer-ppw48" (UID: "ea48fa24-04c4-434f-9cb9-eb941d6fff89") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001081    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-log-socket\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001109    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/82f6bd75-5e49-43df-8d20-b008e485353b-ovnkube-config\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001128    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-system-cni-dir\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001161    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a5e3afa4-7206-4362-9c74-32b04f05d4d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001198    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-var-lib-openvswitch\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001202    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a463d6f0-c349-4a93-8d64-ca9fe04cdc46-konnectivity-ca\") pod \"konnectivity-agent-wl5d4\" (UID: \"a463d6f0-c349-4a93-8d64-ca9fe04cdc46\") " pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001236    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-var-lib-openvswitch\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001304    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-sys-fs\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001361    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbcs6\" (UniqueName: \"kubernetes.io/projected/a5e3afa4-7206-4362-9c74-32b04f05d4d6-kube-api-access-jbcs6\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001391    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea48fa24-04c4-434f-9cb9-eb941d6fff89-dbus\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001398    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-sys-fs\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001416    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2377041-6074-4084-9042-655f433c91ed-hosts-file\") pod \"node-resolver-99b48\" (UID: \"b2377041-6074-4084-9042-655f433c91ed\") " pod="openshift-dns/node-resolver-99b48"
Jul 29 19:17:17.003179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001464    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-slash\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001501    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-etc-openvswitch\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001530    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea48fa24-04c4-434f-9cb9-eb941d6fff89-dbus\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001532    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001545    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b2377041-6074-4084-9042-655f433c91ed-hosts-file\") pod \"node-resolver-99b48\" (UID: \"b2377041-6074-4084-9042-655f433c91ed\") " pod="openshift-dns/node-resolver-99b48"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001570    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d748fef7-568c-4f8c-9698-6f93e239f253-serviceca\") pod \"node-ca-86g76\" (UID: \"d748fef7-568c-4f8c-9698-6f93e239f253\") " pod="openshift-image-registry/node-ca-86g76"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001592    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-slash\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001582    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/49adb559-ddd6-476d-b01d-42e50ca37a83-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001602    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-etc-kubernetes\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001639    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-tuned\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001663    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8sq9\" (UniqueName: \"kubernetes.io/projected/49adb559-ddd6-476d-b01d-42e50ca37a83-kube-api-access-x8sq9\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001702    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-etc-openvswitch\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001736    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-cnibin\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001775    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-os-release\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001798    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-os-release\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001822    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a5e3afa4-7206-4362-9c74-32b04f05d4d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001846    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.003995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001873    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee73b778-04f0-4c42-8769-1fbfbdbab0ab-iptables-alerter-script\") pod \"iptables-alerter-x8qd9\" (UID: \"ee73b778-04f0-4c42-8769-1fbfbdbab0ab\") " pod="openshift-network-operator/iptables-alerter-x8qd9"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001898    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-sysctl-conf\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001922    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/82f6bd75-5e49-43df-8d20-b008e485353b-ovn-node-metrics-cert\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001947    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-run-multus-certs\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001971    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzbp6\" (UniqueName: \"kubernetes.io/projected/f6fd118c-eea4-4850-a41d-849aa67b7448-kube-api-access-jzbp6\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001979    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d748fef7-568c-4f8c-9698-6f93e239f253-serviceca\") pod \"node-ca-86g76\" (UID: \"d748fef7-568c-4f8c-9698-6f93e239f253\") " pod="openshift-image-registry/node-ca-86g76"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.001996    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-modprobe-d\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002025    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-kubernetes\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002050    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/719397e8-732b-4941-bfc3-b1dc71d2371a-tmp\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002073    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qxmv\" (UniqueName: \"kubernetes.io/projected/d748fef7-568c-4f8c-9698-6f93e239f253-kube-api-access-5qxmv\") pod \"node-ca-86g76\" (UID: \"d748fef7-568c-4f8c-9698-6f93e239f253\") " pod="openshift-image-registry/node-ca-86g76"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-var-lib-kubelet\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002137    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d748fef7-568c-4f8c-9698-6f93e239f253-host\") pod \"node-ca-86g76\" (UID: \"d748fef7-568c-4f8c-9698-6f93e239f253\") " pod="openshift-image-registry/node-ca-86g76"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002159    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-sysconfig\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002184    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw6tw\" (UniqueName: \"kubernetes.io/projected/719397e8-732b-4941-bfc3-b1dc71d2371a-kube-api-access-lw6tw\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-kubelet\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-run-netns\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002250    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-sysctl-conf\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.004826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002259    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl2fg\" (UniqueName: \"kubernetes.io/projected/1dcfa88a-bacc-40bc-bb01-fb73b300931d-kube-api-access-vl2fg\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.005573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002301    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d748fef7-568c-4f8c-9698-6f93e239f253-host\") pod \"node-ca-86g76\" (UID: \"d748fef7-568c-4f8c-9698-6f93e239f253\") " pod="openshift-image-registry/node-ca-86g76"
Jul 29 19:17:17.005573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002405    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-sysconfig\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.005573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002531    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/82f6bd75-5e49-43df-8d20-b008e485353b-host-kubelet\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.005573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002610    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ee73b778-04f0-4c42-8769-1fbfbdbab0ab-iptables-alerter-script\") pod \"iptables-alerter-x8qd9\" (UID: \"ee73b778-04f0-4c42-8769-1fbfbdbab0ab\") " pod="openshift-network-operator/iptables-alerter-x8qd9"
Jul 29 19:17:17.005573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002626    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-modprobe-d\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.005573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.002699    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-kubernetes\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.005892 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.005676    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/719397e8-732b-4941-bfc3-b1dc71d2371a-tmp\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.005892 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.005713    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/82f6bd75-5e49-43df-8d20-b008e485353b-ovn-node-metrics-cert\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.005892 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.005871    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/719397e8-732b-4941-bfc3-b1dc71d2371a-etc-tuned\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.006013 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.005969    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a463d6f0-c349-4a93-8d64-ca9fe04cdc46-agent-certs\") pod \"konnectivity-agent-wl5d4\" (UID: \"a463d6f0-c349-4a93-8d64-ca9fe04cdc46\") " pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:17.012383 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.012358    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46ms5\" (UniqueName: \"kubernetes.io/projected/ee73b778-04f0-4c42-8769-1fbfbdbab0ab-kube-api-access-46ms5\") pod \"iptables-alerter-x8qd9\" (UID: \"ee73b778-04f0-4c42-8769-1fbfbdbab0ab\") " pod="openshift-network-operator/iptables-alerter-x8qd9"
Jul 29 19:17:17.013171 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.013109    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zp5f\" (UniqueName: \"kubernetes.io/projected/b2377041-6074-4084-9042-655f433c91ed-kube-api-access-8zp5f\") pod \"node-resolver-99b48\" (UID: \"b2377041-6074-4084-9042-655f433c91ed\") " pod="openshift-dns/node-resolver-99b48"
Jul 29 19:17:17.014291 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.014267    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8sq9\" (UniqueName: \"kubernetes.io/projected/49adb559-ddd6-476d-b01d-42e50ca37a83-kube-api-access-x8sq9\") pod \"aws-ebs-csi-driver-node-9ztj8\" (UID: \"49adb559-ddd6-476d-b01d-42e50ca37a83\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.014561 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.014526    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8246\" (UniqueName: \"kubernetes.io/projected/82f6bd75-5e49-43df-8d20-b008e485353b-kube-api-access-v8246\") pod \"ovnkube-node-ckmtk\" (UID: \"82f6bd75-5e49-43df-8d20-b008e485353b\") " pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.014955 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.014935    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw6tw\" (UniqueName: \"kubernetes.io/projected/719397e8-732b-4941-bfc3-b1dc71d2371a-kube-api-access-lw6tw\") pod \"tuned-ccqx5\" (UID: \"719397e8-732b-4941-bfc3-b1dc71d2371a\") " pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.015440 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.015416    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qxmv\" (UniqueName: \"kubernetes.io/projected/d748fef7-568c-4f8c-9698-6f93e239f253-kube-api-access-5qxmv\") pod \"node-ca-86g76\" (UID: \"d748fef7-568c-4f8c-9698-6f93e239f253\") " pod="openshift-image-registry/node-ca-86g76"
Jul 29 19:17:17.101786 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.101726    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-107.ec2.internal" event={"ID":"ccf7f81c6540eccb14cc3f753f1ba29f","Type":"ContainerStarted","Data":"7a9cd1ffe9beb74789afc689bc8013cedcee87da9728405d933d05d9dba85716"}
Jul 29 19:17:17.102554 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-run-netns\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102679 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl2fg\" (UniqueName: \"kubernetes.io/projected/1dcfa88a-bacc-40bc-bb01-fb73b300931d-kube-api-access-vl2fg\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102679 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102600    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-socket-dir-parent\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102679 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102622    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-var-lib-cni-multus\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102679 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102653    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfk9\" (UniqueName: \"kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9\") pod \"network-check-target-m7mvb\" (UID: \"2db48639-ee08-485b-874c-c64954b418dc\") " pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:17.102679 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-var-lib-cni-bin\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102699    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-daemon-config\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102717    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-run-netns\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-cnibin\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102763    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102792    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-system-cni-dir\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102800    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-var-lib-cni-bin\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102815    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-run-k8s-cni-cncf-io\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102842    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-cni-dir\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102878    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a5e3afa4-7206-4362-9c74-32b04f05d4d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102905    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dcfa88a-bacc-40bc-bb01-fb73b300931d-cni-binary-copy\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102924    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-hostroot\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.102932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102927    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-socket-dir-parent\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102944    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-conf-dir\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102978    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-system-cni-dir\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102766    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-cnibin\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103004    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-cni-dir\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103010    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a5e3afa4-7206-4362-9c74-32b04f05d4d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbcs6\" (UniqueName: \"kubernetes.io/projected/a5e3afa4-7206-4362-9c74-32b04f05d4d6-kube-api-access-jbcs6\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.103064    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103073    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-etc-kubernetes\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103097    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-cnibin\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.103129    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs podName:f6fd118c-eea4-4850-a41d-849aa67b7448 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:17.603108916 +0000 UTC m=+3.098870251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs") pod "network-metrics-daemon-cmpgk" (UID: "f6fd118c-eea4-4850-a41d-849aa67b7448") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103149    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-cnibin\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103148    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-os-release\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103184    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-os-release\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-os-release\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103206    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a5e3afa4-7206-4362-9c74-32b04f05d4d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.102977    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-var-lib-cni-multus\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.103506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103234    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103261    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-run-multus-certs\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103270    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-system-cni-dir\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103284    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzbp6\" (UniqueName: \"kubernetes.io/projected/f6fd118c-eea4-4850-a41d-849aa67b7448-kube-api-access-jzbp6\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103309    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-run-k8s-cni-cncf-io\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103312    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-var-lib-kubelet\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103356    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-conf-dir\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103395    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-system-cni-dir\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103405    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1dcfa88a-bacc-40bc-bb01-fb73b300931d-multus-daemon-config\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103435    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-hostroot\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103497    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-os-release\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103510    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a5e3afa4-7206-4362-9c74-32b04f05d4d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103513    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a5e3afa4-7206-4362-9c74-32b04f05d4d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103569    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-var-lib-kubelet\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103625    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-host-run-multus-certs\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103674    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dcfa88a-bacc-40bc-bb01-fb73b300931d-etc-kubernetes\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.103946    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal" event={"ID":"d2e8713941fec20c0804fb302fd41b7b","Type":"ContainerStarted","Data":"986c8b93449b17cd2a0f16b856b06c13e6bb293c2707204bca326781a9e2dc7f"}
Jul 29 19:17:17.104346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.104096    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dcfa88a-bacc-40bc-bb01-fb73b300931d-cni-binary-copy\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.105299 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.104390    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a5e3afa4-7206-4362-9c74-32b04f05d4d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.105299 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.104392    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a5e3afa4-7206-4362-9c74-32b04f05d4d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.124610 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.124552    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl2fg\" (UniqueName: \"kubernetes.io/projected/1dcfa88a-bacc-40bc-bb01-fb73b300931d-kube-api-access-vl2fg\") pod \"multus-9rhjr\" (UID: \"1dcfa88a-bacc-40bc-bb01-fb73b300931d\") " pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.126131 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.126110    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:17:17.126131 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.126133    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:17:17.126255 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.126147    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5qfk9 for pod openshift-network-diagnostics/network-check-target-m7mvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:17.126255 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.126218    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9 podName:2db48639-ee08-485b-874c-c64954b418dc nodeName:}" failed. No retries permitted until 2026-07-29 19:17:17.626200265 +0000 UTC m=+3.121961588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfk9" (UniqueName: "kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9") pod "network-check-target-m7mvb" (UID: "2db48639-ee08-485b-874c-c64954b418dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:17.133286 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.133261    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbcs6\" (UniqueName: \"kubernetes.io/projected/a5e3afa4-7206-4362-9c74-32b04f05d4d6-kube-api-access-jbcs6\") pod \"multus-additional-cni-plugins-gczs9\" (UID: \"a5e3afa4-7206-4362-9c74-32b04f05d4d6\") " pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.138753 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.138724    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzbp6\" (UniqueName: \"kubernetes.io/projected/f6fd118c-eea4-4850-a41d-849aa67b7448-kube-api-access-jzbp6\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:17.204616 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.204545    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ccqx5"
Jul 29 19:17:17.213524 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.213495    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:17.221580 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.221563    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:17.222258 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.222238    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 19:17:17.229929 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.229909    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x8qd9"
Jul 29 19:17:17.235394 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.235374    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-86g76"
Jul 29 19:17:17.243935 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.243915    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8"
Jul 29 19:17:17.252408 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.252391    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-99b48"
Jul 29 19:17:17.260888 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.260868    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9rhjr"
Jul 29 19:17:17.267433 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.267414    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gczs9"
Jul 29 19:17:17.506027 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.505939    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:17.506191 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.506069    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:17.506191 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.506126    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret podName:ea48fa24-04c4-434f-9cb9-eb941d6fff89 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:18.506111352 +0000 UTC m=+4.001872672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret") pod "global-pull-secret-syncer-ppw48" (UID: "ea48fa24-04c4-434f-9cb9-eb941d6fff89") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:17.607175 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.607142    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:17.607334 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.607301    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:17.607388 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.607369    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs podName:f6fd118c-eea4-4850-a41d-849aa67b7448 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:18.607348929 +0000 UTC m=+4.103110248 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs") pod "network-metrics-daemon-cmpgk" (UID: "f6fd118c-eea4-4850-a41d-849aa67b7448") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:17.707941 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.707900    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfk9\" (UniqueName: \"kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9\") pod \"network-check-target-m7mvb\" (UID: \"2db48639-ee08-485b-874c-c64954b418dc\") " pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:17.708119 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.708079    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:17:17.708119 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.708106    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:17:17.708119 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.708118    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5qfk9 for pod openshift-network-diagnostics/network-check-target-m7mvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:17.708261 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:17.708185    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9 podName:2db48639-ee08-485b-874c-c64954b418dc nodeName:}" failed. No retries permitted until 2026-07-29 19:17:18.708163968 +0000 UTC m=+4.203925311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfk9" (UniqueName: "kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9") pod "network-check-target-m7mvb" (UID: "2db48639-ee08-485b-874c-c64954b418dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:17.943359 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.943272    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 19:12:16 +0000 UTC" deadline="2028-05-13 10:00:11.331213128 +0000 UTC"
Jul 29 19:17:17.943359 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:17.943310    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15686h42m53.387908062s"
Jul 29 19:17:18.096174 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:18.096138    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:18.096357 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:18.096285    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:18.150787 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:18.150758    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda463d6f0_c349_4a93_8d64_ca9fe04cdc46.slice/crio-797c8d1875a8de85e514d6fe7c9fef7b96877d8784702f7537ca1b5067c543c0 WatchSource:0}: Error finding container 797c8d1875a8de85e514d6fe7c9fef7b96877d8784702f7537ca1b5067c543c0: Status 404 returned error can't find the container with id 797c8d1875a8de85e514d6fe7c9fef7b96877d8784702f7537ca1b5067c543c0
Jul 29 19:17:18.152096 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:18.152006    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd748fef7_568c_4f8c_9698_6f93e239f253.slice/crio-a1d0ddb266c3bec71bd198a187731f26e98b2df773d753096eb5042dea8c9b41 WatchSource:0}: Error finding container a1d0ddb266c3bec71bd198a187731f26e98b2df773d753096eb5042dea8c9b41: Status 404 returned error can't find the container with id a1d0ddb266c3bec71bd198a187731f26e98b2df773d753096eb5042dea8c9b41
Jul 29 19:17:18.155214 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:18.155191    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e3afa4_7206_4362_9c74_32b04f05d4d6.slice/crio-9f023574e7c4d0ec4cdcc044d3306a0203c64d1a14ffb1de4e4b1bd8fbe4445d WatchSource:0}: Error finding container 9f023574e7c4d0ec4cdcc044d3306a0203c64d1a14ffb1de4e4b1bd8fbe4445d: Status 404 returned error can't find the container with id 9f023574e7c4d0ec4cdcc044d3306a0203c64d1a14ffb1de4e4b1bd8fbe4445d
Jul 29 19:17:18.156563 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:18.156541    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod719397e8_732b_4941_bfc3_b1dc71d2371a.slice/crio-5b1c095f8885f2711a080ce87d9e0002653a8d637fa7aaf20678dbe1dff4dde8 WatchSource:0}: Error finding container 5b1c095f8885f2711a080ce87d9e0002653a8d637fa7aaf20678dbe1dff4dde8: Status 404 returned error can't find the container with id 5b1c095f8885f2711a080ce87d9e0002653a8d637fa7aaf20678dbe1dff4dde8
Jul 29 19:17:18.158362 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:17:18.158339    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f6bd75_5e49_43df_8d20_b008e485353b.slice/crio-1d9623a4e9ed4b72e70b7933a9da35f20dea406ec768917282439d2b8fa11101 WatchSource:0}: Error finding container 1d9623a4e9ed4b72e70b7933a9da35f20dea406ec768917282439d2b8fa11101: Status 404 returned error can't find the container with id 1d9623a4e9ed4b72e70b7933a9da35f20dea406ec768917282439d2b8fa11101
Jul 29 19:17:18.513254 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:18.513007    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:18.513254 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:18.513173    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:18.513497 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:18.513265    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret podName:ea48fa24-04c4-434f-9cb9-eb941d6fff89 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:20.513246919 +0000 UTC m=+6.009008245 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret") pod "global-pull-secret-syncer-ppw48" (UID: "ea48fa24-04c4-434f-9cb9-eb941d6fff89") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:18.614390 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:18.614334    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:18.614593 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:18.614522    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:18.614593 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:18.614585    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs podName:f6fd118c-eea4-4850-a41d-849aa67b7448 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:20.614566946 +0000 UTC m=+6.110328281 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs") pod "network-metrics-daemon-cmpgk" (UID: "f6fd118c-eea4-4850-a41d-849aa67b7448") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:18.716048 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:18.715665    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfk9\" (UniqueName: \"kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9\") pod \"network-check-target-m7mvb\" (UID: \"2db48639-ee08-485b-874c-c64954b418dc\") " pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:18.716048 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:18.715902    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:17:18.716048 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:18.715924    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:17:18.716048 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:18.715936    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5qfk9 for pod openshift-network-diagnostics/network-check-target-m7mvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:18.716048 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:18.715993    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9 podName:2db48639-ee08-485b-874c-c64954b418dc nodeName:}" failed. No retries permitted until 2026-07-29 19:17:20.715975683 +0000 UTC m=+6.211737005 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfk9" (UniqueName: "kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9") pod "network-check-target-m7mvb" (UID: "2db48639-ee08-485b-874c-c64954b418dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:18.943775 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:18.943528    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 19:12:16 +0000 UTC" deadline="2028-02-19 09:34:36.12507444 +0000 UTC"
Jul 29 19:17:18.943775 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:18.943589    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13670h17m17.181489808s"
Jul 29 19:17:19.097093 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.096566    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:19.097093 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:19.096708    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:19.098270 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.098070    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:19.098270 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:19.098171    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:19.111774 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.111741    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ccqx5" event={"ID":"719397e8-732b-4941-bfc3-b1dc71d2371a","Type":"ContainerStarted","Data":"5b1c095f8885f2711a080ce87d9e0002653a8d637fa7aaf20678dbe1dff4dde8"}
Jul 29 19:17:19.123899 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.123837    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gczs9" event={"ID":"a5e3afa4-7206-4362-9c74-32b04f05d4d6","Type":"ContainerStarted","Data":"9f023574e7c4d0ec4cdcc044d3306a0203c64d1a14ffb1de4e4b1bd8fbe4445d"}
Jul 29 19:17:19.130549 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.130518    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-86g76" event={"ID":"d748fef7-568c-4f8c-9698-6f93e239f253","Type":"ContainerStarted","Data":"a1d0ddb266c3bec71bd198a187731f26e98b2df773d753096eb5042dea8c9b41"}
Jul 29 19:17:19.163030 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.162993    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-107.ec2.internal" event={"ID":"ccf7f81c6540eccb14cc3f753f1ba29f","Type":"ContainerStarted","Data":"3f9c3a615b2d9d9365a024b2c53073417eda8619528afd238a2dc2b98648b5e5"}
Jul 29 19:17:19.170044 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.169903    2566 generic.go:356] "Generic (PLEG): container finished" podID="d2e8713941fec20c0804fb302fd41b7b" containerID="4af196b7e37cd2f0c8894b3123c05b1e984f3ff4c2d3d7f3b8907620269de076" exitCode=0
Jul 29 19:17:19.170044 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.169990    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal" event={"ID":"d2e8713941fec20c0804fb302fd41b7b","Type":"ContainerDied","Data":"4af196b7e37cd2f0c8894b3123c05b1e984f3ff4c2d3d7f3b8907620269de076"}
Jul 29 19:17:19.187543 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.187482    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8" event={"ID":"49adb559-ddd6-476d-b01d-42e50ca37a83","Type":"ContainerStarted","Data":"2a75e9d4e4dec2894c52e9508f9a827086193ca7edb546f1b56083cc8a7399e1"}
Jul 29 19:17:19.196587 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.196019    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-107.ec2.internal" podStartSLOduration=3.196006163 podStartE2EDuration="3.196006163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:19.179397211 +0000 UTC m=+4.675158556" watchObservedRunningTime="2026-07-29 19:17:19.196006163 +0000 UTC m=+4.691767504"
Jul 29 19:17:19.202731 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.202705    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x8qd9" event={"ID":"ee73b778-04f0-4c42-8769-1fbfbdbab0ab","Type":"ContainerStarted","Data":"7112450023fe80ac48288b427079d52e75e26b97618359e8f18e19dec3737ce1"}
Jul 29 19:17:19.217744 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.217698    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9rhjr" event={"ID":"1dcfa88a-bacc-40bc-bb01-fb73b300931d","Type":"ContainerStarted","Data":"3ec12e568f68e28ef2b879c29292a0b8076ea33ad4b0403102d2e05934eb0b19"}
Jul 29 19:17:19.224919 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.224885    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-99b48" event={"ID":"b2377041-6074-4084-9042-655f433c91ed","Type":"ContainerStarted","Data":"188fdb1bd6dc3405c6fdbf2d3164f76900069323890f9e6d19e9ce25739846bb"}
Jul 29 19:17:19.233136 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.233104    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk" event={"ID":"82f6bd75-5e49-43df-8d20-b008e485353b","Type":"ContainerStarted","Data":"1d9623a4e9ed4b72e70b7933a9da35f20dea406ec768917282439d2b8fa11101"}
Jul 29 19:17:19.241984 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:19.241923    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wl5d4" event={"ID":"a463d6f0-c349-4a93-8d64-ca9fe04cdc46","Type":"ContainerStarted","Data":"797c8d1875a8de85e514d6fe7c9fef7b96877d8784702f7537ca1b5067c543c0"}
Jul 29 19:17:20.096193 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:20.096079    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:20.096193 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:20.096181    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:20.276485 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:20.276337    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal" event={"ID":"d2e8713941fec20c0804fb302fd41b7b","Type":"ContainerStarted","Data":"bba1b383f1469ba469b0bd92bc25c369862bbc2cd99f644ed7511a298fc8872f"}
Jul 29 19:17:20.294417 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:20.293806    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-107.ec2.internal" podStartSLOduration=4.293784202 podStartE2EDuration="4.293784202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:20.293185168 +0000 UTC m=+5.788946535" watchObservedRunningTime="2026-07-29 19:17:20.293784202 +0000 UTC m=+5.789545544"
Jul 29 19:17:20.531321 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:20.531232    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:20.531513 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:20.531415    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:20.531513 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:20.531497    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret podName:ea48fa24-04c4-434f-9cb9-eb941d6fff89 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:24.531477407 +0000 UTC m=+10.027238733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret") pod "global-pull-secret-syncer-ppw48" (UID: "ea48fa24-04c4-434f-9cb9-eb941d6fff89") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:20.634603 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:20.632304    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:20.634603 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:20.632505    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:20.634603 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:20.632572    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs podName:f6fd118c-eea4-4850-a41d-849aa67b7448 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:24.632554527 +0000 UTC m=+10.128315847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs") pod "network-metrics-daemon-cmpgk" (UID: "f6fd118c-eea4-4850-a41d-849aa67b7448") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:20.733056 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:20.733016    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfk9\" (UniqueName: \"kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9\") pod \"network-check-target-m7mvb\" (UID: \"2db48639-ee08-485b-874c-c64954b418dc\") " pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:20.733271 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:20.733221    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:17:20.733271 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:20.733239    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:17:20.733271 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:20.733253    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5qfk9 for pod openshift-network-diagnostics/network-check-target-m7mvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:20.733435 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:20.733310    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9 podName:2db48639-ee08-485b-874c-c64954b418dc nodeName:}" failed. No retries permitted until 2026-07-29 19:17:24.733290453 +0000 UTC m=+10.229051776 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfk9" (UniqueName: "kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9") pod "network-check-target-m7mvb" (UID: "2db48639-ee08-485b-874c-c64954b418dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:21.096767 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:21.096724    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:21.097215 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:21.096876    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:21.097215 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:21.096923    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:21.097215 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:21.097047    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:22.096658 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:22.096617    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:22.096836 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:22.096777    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:23.096749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:23.096696    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:23.096932 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:23.096829    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:23.097318 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:23.097165    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:23.097318 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:23.097258    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:24.095772 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:24.095708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:24.095959 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:24.095870    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:24.567905 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:24.567562    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:24.567905 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:24.567722    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:24.567905 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:24.567812    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret podName:ea48fa24-04c4-434f-9cb9-eb941d6fff89 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:32.56779261 +0000 UTC m=+18.063553935 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret") pod "global-pull-secret-syncer-ppw48" (UID: "ea48fa24-04c4-434f-9cb9-eb941d6fff89") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:24.668403 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:24.668367    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:24.668577 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:24.668555    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:24.668700 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:24.668635    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs podName:f6fd118c-eea4-4850-a41d-849aa67b7448 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:32.668615606 +0000 UTC m=+18.164376939 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs") pod "network-metrics-daemon-cmpgk" (UID: "f6fd118c-eea4-4850-a41d-849aa67b7448") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:24.769107 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:24.769069    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfk9\" (UniqueName: \"kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9\") pod \"network-check-target-m7mvb\" (UID: \"2db48639-ee08-485b-874c-c64954b418dc\") " pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:24.769309 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:24.769291    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:17:24.769366 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:24.769316    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:17:24.769366 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:24.769328    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5qfk9 for pod openshift-network-diagnostics/network-check-target-m7mvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:24.769519 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:24.769381    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9 podName:2db48639-ee08-485b-874c-c64954b418dc nodeName:}" failed. No retries permitted until 2026-07-29 19:17:32.769363919 +0000 UTC m=+18.265125245 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfk9" (UniqueName: "kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9") pod "network-check-target-m7mvb" (UID: "2db48639-ee08-485b-874c-c64954b418dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:25.096961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:25.096925    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:25.097141 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:25.097051    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:25.097440 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:25.097415    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:25.097553 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:25.097531    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:26.096479 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:26.096435    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:26.096926 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:26.096591    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:27.095712 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:27.095677    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:27.095712 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:27.095694    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:27.095981 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:27.095825    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:27.095981 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:27.095928    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:28.096332 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:28.096296    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:28.096884 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:28.096494    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:29.096232 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:29.096143    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:29.096393 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:29.096274    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:29.096393 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:29.096329    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:29.096827 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:29.096462    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:30.096170 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:30.096135    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:30.096331 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:30.096270    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:31.096603 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:31.096570    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:31.097048 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:31.096697    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:31.097048 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:31.096757    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:31.097048 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:31.096856    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:32.095601 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:32.095567    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:32.095807 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:32.095700    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:32.632803 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:32.632763    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:32.633230 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:32.632888    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:32.633230 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:32.632966    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret podName:ea48fa24-04c4-434f-9cb9-eb941d6fff89 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:48.632944095 +0000 UTC m=+34.128705437 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret") pod "global-pull-secret-syncer-ppw48" (UID: "ea48fa24-04c4-434f-9cb9-eb941d6fff89") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:32.734070 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:32.734036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:32.734235 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:32.734206    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:32.734363 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:32.734280    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs podName:f6fd118c-eea4-4850-a41d-849aa67b7448 nodeName:}" failed. No retries permitted until 2026-07-29 19:17:48.734261583 +0000 UTC m=+34.230022914 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs") pod "network-metrics-daemon-cmpgk" (UID: "f6fd118c-eea4-4850-a41d-849aa67b7448") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:32.835291 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:32.835252    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfk9\" (UniqueName: \"kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9\") pod \"network-check-target-m7mvb\" (UID: \"2db48639-ee08-485b-874c-c64954b418dc\") " pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:32.835466 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:32.835390    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:17:32.835466 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:32.835407    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:17:32.835466 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:32.835416    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5qfk9 for pod openshift-network-diagnostics/network-check-target-m7mvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:32.835584 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:32.835483    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9 podName:2db48639-ee08-485b-874c-c64954b418dc nodeName:}" failed. No retries permitted until 2026-07-29 19:17:48.835467129 +0000 UTC m=+34.331228468 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfk9" (UniqueName: "kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9") pod "network-check-target-m7mvb" (UID: "2db48639-ee08-485b-874c-c64954b418dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:33.095596 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:33.095515    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:33.095742 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:33.095634    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:33.095742 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:33.095649    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:33.095846 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:33.095760    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:34.095812 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:34.095780    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:34.096176 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:34.095896    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:35.096948 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:35.096915    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:35.097379 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:35.097031    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:35.097379 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:35.097125    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:35.097379 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:35.097024    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:36.096623 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:36.096579    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:36.096780 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:36.096742    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:37.096575 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.096250    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:37.097211 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.096315    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:37.097211 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:37.096665    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:37.097211 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:37.096721    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:37.306415 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.306331    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8" event={"ID":"49adb559-ddd6-476d-b01d-42e50ca37a83","Type":"ContainerStarted","Data":"cd3672ffce2e51e338efdbab39f51302607ffa2997bff1836e18acf01e4df9b4"}
Jul 29 19:17:37.307979 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.307937    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9rhjr" event={"ID":"1dcfa88a-bacc-40bc-bb01-fb73b300931d","Type":"ContainerStarted","Data":"2bf23de6cb3088e47bf52221ee66f3935b1992e014421f37f96f32c79aadb009"}
Jul 29 19:17:37.309534 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.309506    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-99b48" event={"ID":"b2377041-6074-4084-9042-655f433c91ed","Type":"ContainerStarted","Data":"d20e2fc33369964d08d66d6f19a6743115d834cd1ea1f03e9f2ba60b450207f7"}
Jul 29 19:17:37.312787 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.312766    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk" event={"ID":"82f6bd75-5e49-43df-8d20-b008e485353b","Type":"ContainerStarted","Data":"91f7ac276caebad8ee8ac40ef5eda9bb28e1c28be6eb1467f5f8c6e1d76807e1"}
Jul 29 19:17:37.312878 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.312838    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk" event={"ID":"82f6bd75-5e49-43df-8d20-b008e485353b","Type":"ContainerStarted","Data":"b75f1abac105d89599f41bd934d48e7fa9872e8e37ccb93bb513fe5849f1dd97"}
Jul 29 19:17:37.312955 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.312891    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk" event={"ID":"82f6bd75-5e49-43df-8d20-b008e485353b","Type":"ContainerStarted","Data":"e40a77557ba09dff3dbe01c6a63b6e3b3365719607221de37e18d0da0c932094"}
Jul 29 19:17:37.312955 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.312939    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk" event={"ID":"82f6bd75-5e49-43df-8d20-b008e485353b","Type":"ContainerStarted","Data":"7e0ddf939aa05ecd375521fdb15899e5625251e7c7022ce47364654c5e390d76"}
Jul 29 19:17:37.313044 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.312987    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk" event={"ID":"82f6bd75-5e49-43df-8d20-b008e485353b","Type":"ContainerStarted","Data":"dd6949de73048c07aeafb3dab067fca82105bfc84e14288fdd8a4dd4767487e2"}
Jul 29 19:17:37.313044 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.313026    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk" event={"ID":"82f6bd75-5e49-43df-8d20-b008e485353b","Type":"ContainerStarted","Data":"1524f7ad9e897a1560f8b0a23e4a59e21e4a8b51debe1186e6d97a065dada9dd"}
Jul 29 19:17:37.314293 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.314264    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wl5d4" event={"ID":"a463d6f0-c349-4a93-8d64-ca9fe04cdc46","Type":"ContainerStarted","Data":"1b19cf0620338cbe1e8a19660902def38175ee21400ecfd0ed4bed337afd5278"}
Jul 29 19:17:37.315888 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.315861    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ccqx5" event={"ID":"719397e8-732b-4941-bfc3-b1dc71d2371a","Type":"ContainerStarted","Data":"921d243091eb9857deea1e93074f38cf6c0baf3629e75f35d2cecbb0bd27e59e"}
Jul 29 19:17:37.317585 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.317564    2566 generic.go:356] "Generic (PLEG): container finished" podID="a5e3afa4-7206-4362-9c74-32b04f05d4d6" containerID="d35024aa234ed0c00b3a7b9eef086a0a5581bbab87f03a0fd6f6e461fe3ec86b" exitCode=0
Jul 29 19:17:37.317687 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.317645    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gczs9" event={"ID":"a5e3afa4-7206-4362-9c74-32b04f05d4d6","Type":"ContainerDied","Data":"d35024aa234ed0c00b3a7b9eef086a0a5581bbab87f03a0fd6f6e461fe3ec86b"}
Jul 29 19:17:37.319257 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.319233    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-86g76" event={"ID":"d748fef7-568c-4f8c-9698-6f93e239f253","Type":"ContainerStarted","Data":"618373094e2b0f053aea7ba9dde2110fe340df6a93327c1229c972b61383aa9a"}
Jul 29 19:17:37.327989 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.327950    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9rhjr" podStartSLOduration=4.053383448 podStartE2EDuration="22.327936209s" totalImagesPullingTime="18.274552761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:37.327548515 +0000 UTC m=+22.823309883" watchObservedRunningTime="2026-07-29 19:17:37.327936209 +0000 UTC m=+22.823697553"
Jul 29 19:17:37.342067 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.342033    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wl5d4" podStartSLOduration=4.118964682 podStartE2EDuration="22.34202136s" totalImagesPullingTime="18.223056678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:37.341468946 +0000 UTC m=+22.837230306" watchObservedRunningTime="2026-07-29 19:17:37.34202136 +0000 UTC m=+22.837782703"
Jul 29 19:17:37.356317 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.356285    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-86g76" podStartSLOduration=4.127299441 podStartE2EDuration="22.356273514s" totalImagesPullingTime="18.228974073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:37.355747004 +0000 UTC m=+22.851508373" watchObservedRunningTime="2026-07-29 19:17:37.356273514 +0000 UTC m=+22.852034855"
Jul 29 19:17:37.373728 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.373680    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-99b48" podStartSLOduration=4.156478997 podStartE2EDuration="22.373665512s" totalImagesPullingTime="18.217186515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:37.369753413 +0000 UTC m=+22.865514745" watchObservedRunningTime="2026-07-29 19:17:37.373665512 +0000 UTC m=+22.869426855"
Jul 29 19:17:37.407811 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.407764    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ccqx5" podStartSLOduration=4.190881753 podStartE2EDuration="22.407749173s" totalImagesPullingTime="18.21686742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:37.407219888 +0000 UTC m=+22.902981230" watchObservedRunningTime="2026-07-29 19:17:37.407749173 +0000 UTC m=+22.903510516"
Jul 29 19:17:37.960024 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.960002    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 19:17:37.962114 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.962032    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T19:17:37.960019154Z","UUID":"e2809ddd-0cad-4fb6-b939-32263a7ec24c","Handler":null,"Name":"","Endpoint":""}
Jul 29 19:17:37.963256 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.963237    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 19:17:37.963352 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:37.963272    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 19:17:38.096434 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:38.096408    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:38.096642 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:38.096536    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:38.323489 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:38.323385    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x8qd9" event={"ID":"ee73b778-04f0-4c42-8769-1fbfbdbab0ab","Type":"ContainerStarted","Data":"bc5ada56cfdb2dac032e794779b3b8eaef481cda6a426262459f5f72b0c4fb1d"}
Jul 29 19:17:38.325374 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:38.325345    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8" event={"ID":"49adb559-ddd6-476d-b01d-42e50ca37a83","Type":"ContainerStarted","Data":"3f4f87f84e16096571ed5782790c7c966ba4498e85dbec5c37386bd6518060d5"}
Jul 29 19:17:38.339631 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:38.339585    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x8qd9" podStartSLOduration=5.116080407 podStartE2EDuration="23.339567628s" totalImagesPullingTime="18.223487221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:38.339375342 +0000 UTC m=+23.835136698" watchObservedRunningTime="2026-07-29 19:17:38.339567628 +0000 UTC m=+23.835328971"
Jul 29 19:17:39.096241 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:39.096146    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:39.096241 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:39.096166    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:39.096513 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:39.096283    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:39.096513 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:39.096397    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:39.518010 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:39.517973    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:39.518921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:39.518900    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:40.096490 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:40.096446    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:40.096675 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:40.096601    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:40.333500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:40.333466    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk" event={"ID":"82f6bd75-5e49-43df-8d20-b008e485353b","Type":"ContainerStarted","Data":"369844b62049efbae347c0de013e5ac39e4e7412169d71d8bb9ca49a7dd9aeb2"}
Jul 29 19:17:40.335669 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:40.335638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8" event={"ID":"49adb559-ddd6-476d-b01d-42e50ca37a83","Type":"ContainerStarted","Data":"143449afb065ad4b88837367c3b87f0c372b6a78795c90d6209a4fa668b14bc5"}
Jul 29 19:17:40.335891 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:40.335860    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:40.336628 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:40.336603    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wl5d4"
Jul 29 19:17:40.369237 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:40.369161    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9ztj8" podStartSLOduration=4.252767441 podStartE2EDuration="25.369141685s" totalImagesPullingTime="21.116374244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:40.353352383 +0000 UTC m=+25.849113747" watchObservedRunningTime="2026-07-29 19:17:40.369141685 +0000 UTC m=+25.864903029"
Jul 29 19:17:41.096143 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:41.096100    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:41.096570 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:41.096236    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:41.096570 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:41.096289    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:41.096570 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:41.096399    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:42.096269 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:42.096101    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:42.096960 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:42.096344    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:42.342797 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:42.342765    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk" event={"ID":"82f6bd75-5e49-43df-8d20-b008e485353b","Type":"ContainerStarted","Data":"adb3c76e66555817a476817f8ac351a9b148e276f6728484f2f4ddf7a92129a5"}
Jul 29 19:17:42.343027 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:42.343007    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:42.343105 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:42.343080    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:42.343151 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:42.343125    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:42.344806 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:42.344786    2566 generic.go:356] "Generic (PLEG): container finished" podID="a5e3afa4-7206-4362-9c74-32b04f05d4d6" containerID="2c7c369c9eae4296336d360c0384f2cf01f453cf47db84dda5f9f8ea9f331f0d" exitCode=0
Jul 29 19:17:42.344886 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:42.344868    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gczs9" event={"ID":"a5e3afa4-7206-4362-9c74-32b04f05d4d6","Type":"ContainerDied","Data":"2c7c369c9eae4296336d360c0384f2cf01f453cf47db84dda5f9f8ea9f331f0d"}
Jul 29 19:17:42.358519 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:42.358498    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:42.358643 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:42.358632    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:17:42.371555 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:42.371504    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk" podStartSLOduration=9.080289839 podStartE2EDuration="27.371492066s" totalImagesPullingTime="18.291202227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:42.370772794 +0000 UTC m=+27.866534136" watchObservedRunningTime="2026-07-29 19:17:42.371492066 +0000 UTC m=+27.867253408"
Jul 29 19:17:43.096297 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:43.096059    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:43.096741 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:43.096123    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:43.096741 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:43.096348    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:43.096741 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:43.096413    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:43.927071 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:43.927041    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ppw48"]
Jul 29 19:17:43.927248 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:43.927154    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:43.927309 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:43.927265    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:43.930043 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:43.930013    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m7mvb"]
Jul 29 19:17:43.930159 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:43.930113    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:43.930240 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:43.930215    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:43.930626 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:43.930597    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cmpgk"]
Jul 29 19:17:43.930709 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:43.930697    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:43.930803 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:43.930781    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:44.350748 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:44.350719    2566 generic.go:356] "Generic (PLEG): container finished" podID="a5e3afa4-7206-4362-9c74-32b04f05d4d6" containerID="224f361b3a859f4c460ae8808b3caa7b325ca079db53f625fd9ed9b342872f70" exitCode=0
Jul 29 19:17:44.351191 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:44.350799    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gczs9" event={"ID":"a5e3afa4-7206-4362-9c74-32b04f05d4d6","Type":"ContainerDied","Data":"224f361b3a859f4c460ae8808b3caa7b325ca079db53f625fd9ed9b342872f70"}
Jul 29 19:17:46.096787 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:46.096562    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:46.097160 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:46.096563    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:46.097160 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:46.096802    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:46.097160 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:46.096604    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:46.097160 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:46.096873    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:46.097160 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:46.096956    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:46.357773 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:46.357680    2566 generic.go:356] "Generic (PLEG): container finished" podID="a5e3afa4-7206-4362-9c74-32b04f05d4d6" containerID="22a94e78b2f040d1760e6427f4a279ad2c646362dfcd824bd407af6ac0c7a7ed" exitCode=0
Jul 29 19:17:46.357773 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:46.357720    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gczs9" event={"ID":"a5e3afa4-7206-4362-9c74-32b04f05d4d6","Type":"ContainerDied","Data":"22a94e78b2f040d1760e6427f4a279ad2c646362dfcd824bd407af6ac0c7a7ed"}
Jul 29 19:17:46.662053 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:46.661985    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-99b48_b2377041-6074-4084-9042-655f433c91ed/dns-node-resolver/0.log"
Jul 29 19:17:47.635207 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:47.635177    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-86g76_d748fef7-568c-4f8c-9698-6f93e239f253/node-ca/0.log"
Jul 29 19:17:48.095801 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:48.095771    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:48.095952 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:48.095778    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:48.095952 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:48.095881    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:48.096052 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:48.095776    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:48.096052 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:48.095964    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:48.096052 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:48.096037    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:48.662972 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:48.662932    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:48.663423 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:48.663078    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:48.663423 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:48.663148    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret podName:ea48fa24-04c4-434f-9cb9-eb941d6fff89 nodeName:}" failed. No retries permitted until 2026-07-29 19:18:20.663127788 +0000 UTC m=+66.158889127 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret") pod "global-pull-secret-syncer-ppw48" (UID: "ea48fa24-04c4-434f-9cb9-eb941d6fff89") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:17:48.764348 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:48.764311    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:48.764516 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:48.764494    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:48.764597 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:48.764570    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs podName:f6fd118c-eea4-4850-a41d-849aa67b7448 nodeName:}" failed. No retries permitted until 2026-07-29 19:18:20.764549945 +0000 UTC m=+66.260311280 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs") pod "network-metrics-daemon-cmpgk" (UID: "f6fd118c-eea4-4850-a41d-849aa67b7448") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:17:48.865575 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:48.865537    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfk9\" (UniqueName: \"kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9\") pod \"network-check-target-m7mvb\" (UID: \"2db48639-ee08-485b-874c-c64954b418dc\") " pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:48.865726 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:48.865679    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:17:48.865726 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:48.865704    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:17:48.865726 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:48.865717    2566 projected.go:196] Error preparing data for projected volume kube-api-access-5qfk9 for pod openshift-network-diagnostics/network-check-target-m7mvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:48.865844 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:48.865769    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9 podName:2db48639-ee08-485b-874c-c64954b418dc nodeName:}" failed. No retries permitted until 2026-07-29 19:18:20.865756487 +0000 UTC m=+66.361517807 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfk9" (UniqueName: "kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9") pod "network-check-target-m7mvb" (UID: "2db48639-ee08-485b-874c-c64954b418dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:17:50.095926 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:50.095896    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:50.096469 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:50.095895    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:50.096469 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:50.096012    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:50.096469 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:50.095895    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:50.096469 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:50.096162    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:50.096469 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:50.096241    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:50.370478 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:50.369792    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gczs9" event={"ID":"a5e3afa4-7206-4362-9c74-32b04f05d4d6","Type":"ContainerStarted","Data":"f856362cedd056b2139053d0ad0501956e202cdafe3ac5d53bdbc6d11ae4a670"}
Jul 29 19:17:51.374233 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:51.374202    2566 generic.go:356] "Generic (PLEG): container finished" podID="a5e3afa4-7206-4362-9c74-32b04f05d4d6" containerID="f856362cedd056b2139053d0ad0501956e202cdafe3ac5d53bdbc6d11ae4a670" exitCode=0
Jul 29 19:17:51.374233 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:51.374239    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gczs9" event={"ID":"a5e3afa4-7206-4362-9c74-32b04f05d4d6","Type":"ContainerDied","Data":"f856362cedd056b2139053d0ad0501956e202cdafe3ac5d53bdbc6d11ae4a670"}
Jul 29 19:17:52.095707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:52.095682    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:52.095855 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:52.095688    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:52.095855 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:52.095774    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:52.095855 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:52.095790    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:52.095954 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:52.095875    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:52.095987 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:52.095954    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:52.379370 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:52.379296    2566 generic.go:356] "Generic (PLEG): container finished" podID="a5e3afa4-7206-4362-9c74-32b04f05d4d6" containerID="cd2445e7a66e9621ce34ce931543406e80700be9921a8f18591d3c6e6be313c0" exitCode=0
Jul 29 19:17:52.379370 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:52.379336    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gczs9" event={"ID":"a5e3afa4-7206-4362-9c74-32b04f05d4d6","Type":"ContainerDied","Data":"cd2445e7a66e9621ce34ce931543406e80700be9921a8f18591d3c6e6be313c0"}
Jul 29 19:17:53.385043 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:53.385011    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gczs9" event={"ID":"a5e3afa4-7206-4362-9c74-32b04f05d4d6","Type":"ContainerStarted","Data":"6e0593195e1a2592aa9856074edfff127c5c757c9204c451cd5914a4174ecf0f"}
Jul 29 19:17:53.412050 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:53.412010    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gczs9" podStartSLOduration=9.603577009 podStartE2EDuration="38.411996022s" totalImagesPullingTime="28.808419013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:17:53.410252176 +0000 UTC m=+38.906013518" watchObservedRunningTime="2026-07-29 19:17:53.411996022 +0000 UTC m=+38.907757364"
Jul 29 19:17:54.096161 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:54.096130    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:54.096161 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:54.096145    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:54.096376 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:54.096130    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:54.096376 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:54.096238    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:54.096376 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:54.096294    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:54.096560 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:54.096403    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:56.095973 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:56.095941    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:56.095973 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:56.095975    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:56.096363 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:56.096048    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:56.096363 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:56.096059    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:17:56.096363 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:56.096108    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:56.096363 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:56.096194    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:58.096093 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:58.096062    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:17:58.096579 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:58.096163    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:17:58.096579 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:58.096171    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:17:58.096579 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:17:58.096192    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:17:58.096579 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:58.096275    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:17:58.096579 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:17:58.096372    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:18:00.096144 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:00.096113    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:18:00.096620 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:00.096582    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:18:00.096688 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:00.096628    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:18:00.098947 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:00.097033    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:18:00.098947 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:00.097215    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:18:00.098947 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:00.097342    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:18:02.095739 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:02.095710    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:18:02.096121 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:02.095710    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:18:02.096121 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:02.095710    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:18:02.096121 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:02.095891    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:18:02.096121 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:02.095801    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:18:02.096121 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:02.095967    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:18:04.096221 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:04.096184    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:18:04.096221 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:04.096215    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:18:04.096744 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:04.096229    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:18:04.096744 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:04.096309    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:18:04.096744 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:04.096426    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:18:04.096744 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:04.096538    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:18:06.095735 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:06.095708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:18:06.096187 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:06.095708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:18:06.096187 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:06.095803    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:18:06.096187 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:06.095708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:18:06.096187 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:06.095934    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:18:06.096187 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:06.095871    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:18:08.095913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:08.095886    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:18:08.095913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:08.095917    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:18:08.096388 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:08.095885    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:18:08.096388 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:08.095989    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:18:08.096388 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:08.096050    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:18:08.096388 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:08.096124    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:18:10.095789 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.095747    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:18:10.095789 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.095782    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:18:10.096183 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:10.095870    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7mvb" podUID="2db48639-ee08-485b-874c-c64954b418dc"
Jul 29 19:18:10.096183 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.095922    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:18:10.096183 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:10.095999    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ppw48" podUID="ea48fa24-04c4-434f-9cb9-eb941d6fff89"
Jul 29 19:18:10.096183 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:10.096087    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmpgk" podUID="f6fd118c-eea4-4850-a41d-849aa67b7448"
Jul 29 19:18:10.320209 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.320178    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-107.ec2.internal" event="NodeReady"
Jul 29 19:18:10.320393 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.320315    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 19:18:10.355817 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.355790    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7756dbb7c9-wvsz8"]
Jul 29 19:18:10.357573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.357557    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.360494 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.360475    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xs7xs\""
Jul 29 19:18:10.360809 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.360789    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 19:18:10.360997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.360971    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 19:18:10.361081 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.361012    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 19:18:10.366095 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.366079    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 19:18:10.368359 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.368340    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dbr22"]
Jul 29 19:18:10.369986 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.369971    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.371193 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.371172    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bfvhx"]
Jul 29 19:18:10.372556 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.372540    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 19:18:10.372556 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.372550    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 19:18:10.372682 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.372562    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hhdrq\""
Jul 29 19:18:10.372935 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.372914    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.374418 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.374398    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7756dbb7c9-wvsz8"]
Jul 29 19:18:10.375569 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.375549    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 19:18:10.375712 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.375592    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nb5jk\""
Jul 29 19:18:10.375712 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.375647    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 19:18:10.375894 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.375879    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 19:18:10.375937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.375908    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 19:18:10.382187 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.382167    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dbr22"]
Jul 29 19:18:10.388587 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.388568    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bfvhx"]
Jul 29 19:18:10.394546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.394529    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ctkzl"]
Jul 29 19:18:10.396231 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.396218    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ctkzl"
Jul 29 19:18:10.398870 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.398850    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 19:18:10.398953 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.398870    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2xxt4\""
Jul 29 19:18:10.398953 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.398850    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 19:18:10.399266 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.399251    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 19:18:10.407866 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.407848    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ctkzl"]
Jul 29 19:18:10.435118 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435099    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b17d923e-a110-4e2a-9c0c-fa0e2d664199-registry-tls\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.435226 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435125    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-tmp-dir\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.435226 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435158    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ztqv\" (UniqueName: \"kubernetes.io/projected/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-kube-api-access-2ztqv\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.435306 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435230    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b17d923e-a110-4e2a-9c0c-fa0e2d664199-bound-sa-token\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.435306 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435262    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78g7z\" (UniqueName: \"kubernetes.io/projected/b17d923e-a110-4e2a-9c0c-fa0e2d664199-kube-api-access-78g7z\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.435306 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435291    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e430e745-31c3-48f0-bfaf-ce61e230f95a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.435427 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435326    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/553abb5a-4c65-47ab-ad96-bfe1e231265c-cert\") pod \"ingress-canary-ctkzl\" (UID: \"553abb5a-4c65-47ab-ad96-bfe1e231265c\") " pod="openshift-ingress-canary/ingress-canary-ctkzl"
Jul 29 19:18:10.435427 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435357    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b17d923e-a110-4e2a-9c0c-fa0e2d664199-trusted-ca\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.435427 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435389    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-metrics-tls\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.435427 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435410    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b17d923e-a110-4e2a-9c0c-fa0e2d664199-registry-certificates\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.435617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435436    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e430e745-31c3-48f0-bfaf-ce61e230f95a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.435617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435482    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wnm9\" (UniqueName: \"kubernetes.io/projected/e430e745-31c3-48f0-bfaf-ce61e230f95a-kube-api-access-5wnm9\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.435617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435499    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbjmh\" (UniqueName: \"kubernetes.io/projected/553abb5a-4c65-47ab-ad96-bfe1e231265c-kube-api-access-bbjmh\") pod \"ingress-canary-ctkzl\" (UID: \"553abb5a-4c65-47ab-ad96-bfe1e231265c\") " pod="openshift-ingress-canary/ingress-canary-ctkzl"
Jul 29 19:18:10.435617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435518    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b17d923e-a110-4e2a-9c0c-fa0e2d664199-image-registry-private-configuration\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.435617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435535    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b17d923e-a110-4e2a-9c0c-fa0e2d664199-installation-pull-secrets\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.435617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435553    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e430e745-31c3-48f0-bfaf-ce61e230f95a-crio-socket\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.435617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435570    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b17d923e-a110-4e2a-9c0c-fa0e2d664199-ca-trust-extracted\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.435617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435585    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-config-volume\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.435617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.435608    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e430e745-31c3-48f0-bfaf-ce61e230f95a-data-volume\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.536841 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.536813    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e430e745-31c3-48f0-bfaf-ce61e230f95a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.536841 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.536841    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wnm9\" (UniqueName: \"kubernetes.io/projected/e430e745-31c3-48f0-bfaf-ce61e230f95a-kube-api-access-5wnm9\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.537020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.536857    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbjmh\" (UniqueName: \"kubernetes.io/projected/553abb5a-4c65-47ab-ad96-bfe1e231265c-kube-api-access-bbjmh\") pod \"ingress-canary-ctkzl\" (UID: \"553abb5a-4c65-47ab-ad96-bfe1e231265c\") " pod="openshift-ingress-canary/ingress-canary-ctkzl"
Jul 29 19:18:10.537020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.536875    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b17d923e-a110-4e2a-9c0c-fa0e2d664199-image-registry-private-configuration\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.537020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.536907    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b17d923e-a110-4e2a-9c0c-fa0e2d664199-installation-pull-secrets\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.537020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.536933    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e430e745-31c3-48f0-bfaf-ce61e230f95a-crio-socket\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.537020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.536959    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b17d923e-a110-4e2a-9c0c-fa0e2d664199-ca-trust-extracted\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.537020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.536985    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-config-volume\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.537312 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537024    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e430e745-31c3-48f0-bfaf-ce61e230f95a-data-volume\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.537312 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e430e745-31c3-48f0-bfaf-ce61e230f95a-crio-socket\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.537312 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537250    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b17d923e-a110-4e2a-9c0c-fa0e2d664199-registry-tls\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.537312 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537278    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-tmp-dir\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.537537 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537320    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ztqv\" (UniqueName: \"kubernetes.io/projected/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-kube-api-access-2ztqv\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.537537 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537350    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b17d923e-a110-4e2a-9c0c-fa0e2d664199-bound-sa-token\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.537537 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537374    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78g7z\" (UniqueName: \"kubernetes.io/projected/b17d923e-a110-4e2a-9c0c-fa0e2d664199-kube-api-access-78g7z\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.537537 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537399    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e430e745-31c3-48f0-bfaf-ce61e230f95a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.537537 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537439    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/553abb5a-4c65-47ab-ad96-bfe1e231265c-cert\") pod \"ingress-canary-ctkzl\" (UID: \"553abb5a-4c65-47ab-ad96-bfe1e231265c\") " pod="openshift-ingress-canary/ingress-canary-ctkzl"
Jul 29 19:18:10.537537 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537474    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b17d923e-a110-4e2a-9c0c-fa0e2d664199-ca-trust-extracted\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.537537 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537498    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b17d923e-a110-4e2a-9c0c-fa0e2d664199-trusted-ca\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.537537 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537537    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-metrics-tls\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.537921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537548    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e430e745-31c3-48f0-bfaf-ce61e230f95a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.537921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537566    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b17d923e-a110-4e2a-9c0c-fa0e2d664199-registry-certificates\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.537921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537607    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-config-volume\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.537921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.537433    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e430e745-31c3-48f0-bfaf-ce61e230f95a-data-volume\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.538329 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.538240    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-tmp-dir\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.538329 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.538270    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b17d923e-a110-4e2a-9c0c-fa0e2d664199-registry-certificates\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.539087 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.539065    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b17d923e-a110-4e2a-9c0c-fa0e2d664199-trusted-ca\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.541359 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.541331    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e430e745-31c3-48f0-bfaf-ce61e230f95a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.541485 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.541334    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-metrics-tls\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.541567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.541491    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b17d923e-a110-4e2a-9c0c-fa0e2d664199-registry-tls\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.541567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.541517    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b17d923e-a110-4e2a-9c0c-fa0e2d664199-image-registry-private-configuration\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.541567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.541535    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b17d923e-a110-4e2a-9c0c-fa0e2d664199-installation-pull-secrets\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.541699 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.541629    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/553abb5a-4c65-47ab-ad96-bfe1e231265c-cert\") pod \"ingress-canary-ctkzl\" (UID: \"553abb5a-4c65-47ab-ad96-bfe1e231265c\") " pod="openshift-ingress-canary/ingress-canary-ctkzl"
Jul 29 19:18:10.545856 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.545829    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wnm9\" (UniqueName: \"kubernetes.io/projected/e430e745-31c3-48f0-bfaf-ce61e230f95a-kube-api-access-5wnm9\") pod \"insights-runtime-extractor-bfvhx\" (UID: \"e430e745-31c3-48f0-bfaf-ce61e230f95a\") " pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.546019 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.546001    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ztqv\" (UniqueName: \"kubernetes.io/projected/ec82c59a-a7ae-46f8-884d-b3ad641bdb97-kube-api-access-2ztqv\") pod \"dns-default-dbr22\" (UID: \"ec82c59a-a7ae-46f8-884d-b3ad641bdb97\") " pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.548011 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.547987    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbjmh\" (UniqueName: \"kubernetes.io/projected/553abb5a-4c65-47ab-ad96-bfe1e231265c-kube-api-access-bbjmh\") pod \"ingress-canary-ctkzl\" (UID: \"553abb5a-4c65-47ab-ad96-bfe1e231265c\") " pod="openshift-ingress-canary/ingress-canary-ctkzl"
Jul 29 19:18:10.548149 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.548121    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b17d923e-a110-4e2a-9c0c-fa0e2d664199-bound-sa-token\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.548702 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.548682    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78g7z\" (UniqueName: \"kubernetes.io/projected/b17d923e-a110-4e2a-9c0c-fa0e2d664199-kube-api-access-78g7z\") pod \"image-registry-7756dbb7c9-wvsz8\" (UID: \"b17d923e-a110-4e2a-9c0c-fa0e2d664199\") " pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.667687 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.667610    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:10.680625 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.680600    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:10.685491 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.685472    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bfvhx"
Jul 29 19:18:10.704437 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.704393    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ctkzl"
Jul 29 19:18:10.847492 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.847464    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dbr22"]
Jul 29 19:18:10.851484 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:10.851440    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec82c59a_a7ae_46f8_884d_b3ad641bdb97.slice/crio-fcb15d9d0238e4968ca43ce7dcd1a71dcac1508bbbc649224e6ab43a2dbcbd92 WatchSource:0}: Error finding container fcb15d9d0238e4968ca43ce7dcd1a71dcac1508bbbc649224e6ab43a2dbcbd92: Status 404 returned error can't find the container with id fcb15d9d0238e4968ca43ce7dcd1a71dcac1508bbbc649224e6ab43a2dbcbd92
Jul 29 19:18:10.872503 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.872482    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7756dbb7c9-wvsz8"]
Jul 29 19:18:10.874391 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:10.874370    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb17d923e_a110_4e2a_9c0c_fa0e2d664199.slice/crio-4fb0f31cf446e1732dda9eb9211bbe772be2efd74d124762fb1e2f74d263766b WatchSource:0}: Error finding container 4fb0f31cf446e1732dda9eb9211bbe772be2efd74d124762fb1e2f74d263766b: Status 404 returned error can't find the container with id 4fb0f31cf446e1732dda9eb9211bbe772be2efd74d124762fb1e2f74d263766b
Jul 29 19:18:10.881198 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.881178    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bfvhx"]
Jul 29 19:18:10.884542 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:10.884520    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ctkzl"]
Jul 29 19:18:10.885372 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:10.885347    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode430e745_31c3_48f0_bfaf_ce61e230f95a.slice/crio-502e0a89b6db45ab2af9713810a1fe5fa55c9427e9f65b776835b62aebb18d11 WatchSource:0}: Error finding container 502e0a89b6db45ab2af9713810a1fe5fa55c9427e9f65b776835b62aebb18d11: Status 404 returned error can't find the container with id 502e0a89b6db45ab2af9713810a1fe5fa55c9427e9f65b776835b62aebb18d11
Jul 29 19:18:10.888677 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:10.887136    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod553abb5a_4c65_47ab_ad96_bfe1e231265c.slice/crio-018b17803559878d478702f420247d98796dba9e02c111cdbdc98eaaa89c5b5b WatchSource:0}: Error finding container 018b17803559878d478702f420247d98796dba9e02c111cdbdc98eaaa89c5b5b: Status 404 returned error can't find the container with id 018b17803559878d478702f420247d98796dba9e02c111cdbdc98eaaa89c5b5b
Jul 29 19:18:11.427230 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:11.427156    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ctkzl" event={"ID":"553abb5a-4c65-47ab-ad96-bfe1e231265c","Type":"ContainerStarted","Data":"018b17803559878d478702f420247d98796dba9e02c111cdbdc98eaaa89c5b5b"}
Jul 29 19:18:11.432813 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:11.431893    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8" event={"ID":"b17d923e-a110-4e2a-9c0c-fa0e2d664199","Type":"ContainerStarted","Data":"2888aa8a9fc4f1dbd6016f17ef4b3268205c23ddb4d585103f7c0de1965cb59c"}
Jul 29 19:18:11.432813 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:11.431938    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8" event={"ID":"b17d923e-a110-4e2a-9c0c-fa0e2d664199","Type":"ContainerStarted","Data":"4fb0f31cf446e1732dda9eb9211bbe772be2efd74d124762fb1e2f74d263766b"}
Jul 29 19:18:11.432813 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:11.432677    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:11.434324 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:11.434244    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dbr22" event={"ID":"ec82c59a-a7ae-46f8-884d-b3ad641bdb97","Type":"ContainerStarted","Data":"fcb15d9d0238e4968ca43ce7dcd1a71dcac1508bbbc649224e6ab43a2dbcbd92"}
Jul 29 19:18:11.436439 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:11.436419    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bfvhx" event={"ID":"e430e745-31c3-48f0-bfaf-ce61e230f95a","Type":"ContainerStarted","Data":"502e0a89b6db45ab2af9713810a1fe5fa55c9427e9f65b776835b62aebb18d11"}
Jul 29 19:18:11.455111 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:11.454508    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8" podStartSLOduration=2.454490331 podStartE2EDuration="2.454490331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:11.45304221 +0000 UTC m=+56.948803584" watchObservedRunningTime="2026-07-29 19:18:11.454490331 +0000 UTC m=+56.950251651"
Jul 29 19:18:12.095675 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:12.095608    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:18:12.095675 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:12.095634    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:18:12.096432 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:12.096154    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:18:12.100182 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:12.100161    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 19:18:12.100607 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:12.100369    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qxkk6\""
Jul 29 19:18:12.100607 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:12.100583    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 19:18:12.100786 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:12.100677    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 19:18:12.100845 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:12.100805    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b2qz7\""
Jul 29 19:18:12.101075 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:12.100911    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 19:18:13.631194 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:13.631160    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"]
Jul 29 19:18:13.633042 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:13.633023    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"
Jul 29 19:18:13.635707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:13.635684    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 29 19:18:13.635839 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:13.635813    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8t52b\""
Jul 29 19:18:13.644650 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:13.644616    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"]
Jul 29 19:18:13.664610 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:13.664578    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8b31b5d7-12fd-4b25-8876-430d3d6cf34b-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-5gs8s\" (UID: \"8b31b5d7-12fd-4b25-8876-430d3d6cf34b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"
Jul 29 19:18:13.765179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:13.765127    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8b31b5d7-12fd-4b25-8876-430d3d6cf34b-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-5gs8s\" (UID: \"8b31b5d7-12fd-4b25-8876-430d3d6cf34b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"
Jul 29 19:18:13.765332 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:13.765285    2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 19:18:13.765382 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:13.765357    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b31b5d7-12fd-4b25-8876-430d3d6cf34b-tls-certificates podName:8b31b5d7-12fd-4b25-8876-430d3d6cf34b nodeName:}" failed. No retries permitted until 2026-07-29 19:18:14.265337953 +0000 UTC m=+59.761099292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/8b31b5d7-12fd-4b25-8876-430d3d6cf34b-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-5gs8s" (UID: "8b31b5d7-12fd-4b25-8876-430d3d6cf34b") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 19:18:14.269081 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:14.269039    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8b31b5d7-12fd-4b25-8876-430d3d6cf34b-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-5gs8s\" (UID: \"8b31b5d7-12fd-4b25-8876-430d3d6cf34b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"
Jul 29 19:18:14.269270 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:14.269191    2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 19:18:14.269270 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:14.269259    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b31b5d7-12fd-4b25-8876-430d3d6cf34b-tls-certificates podName:8b31b5d7-12fd-4b25-8876-430d3d6cf34b nodeName:}" failed. No retries permitted until 2026-07-29 19:18:15.269243573 +0000 UTC m=+60.765004892 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/8b31b5d7-12fd-4b25-8876-430d3d6cf34b-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-5gs8s" (UID: "8b31b5d7-12fd-4b25-8876-430d3d6cf34b") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 19:18:14.365562 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:14.365534    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ckmtk"
Jul 29 19:18:15.279354 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.279310    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8b31b5d7-12fd-4b25-8876-430d3d6cf34b-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-5gs8s\" (UID: \"8b31b5d7-12fd-4b25-8876-430d3d6cf34b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"
Jul 29 19:18:15.282031 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.282000    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8b31b5d7-12fd-4b25-8876-430d3d6cf34b-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-5gs8s\" (UID: \"8b31b5d7-12fd-4b25-8876-430d3d6cf34b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"
Jul 29 19:18:15.445690 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.445662    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8t52b\""
Jul 29 19:18:15.452931 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.452891    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"
Jul 29 19:18:15.551420 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.551338    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-846bf86cb7-ktvjs"]
Jul 29 19:18:15.785611 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.785588    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-846bf86cb7-ktvjs"]
Jul 29 19:18:15.785730 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.785681    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.788724 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.788699    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8pjf9\""
Jul 29 19:18:15.788832 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.788754    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 19:18:15.788832 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.788774    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 19:18:15.789834 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.789812    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 19:18:15.789834 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.789812    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 19:18:15.789962 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.789857    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 19:18:15.789962 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.789858    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 19:18:15.790062 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.790047    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 19:18:15.884881 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.884824    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-oauth-serving-cert\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.884881 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.884855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-serving-cert\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.885011 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.884901    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-service-ca\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.885011 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.884941    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-oauth-config\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.885011 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.884958    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcmrc\" (UniqueName: \"kubernetes.io/projected/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-kube-api-access-mcmrc\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.885113 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.885022    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-config\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.985517 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.985439    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-oauth-config\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.985666 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.985537    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcmrc\" (UniqueName: \"kubernetes.io/projected/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-kube-api-access-mcmrc\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.985666 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.985564    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-config\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.985666 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.985590    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-oauth-serving-cert\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.985666 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.985617    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-serving-cert\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.985852 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.985679    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-service-ca\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.986272 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.986251    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-oauth-serving-cert\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.986357 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.986272    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-config\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.986357 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.986274    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-service-ca\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.987762 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.987740    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-oauth-config\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.987883 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.987863    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-serving-cert\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:15.994467 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:15.994432    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcmrc\" (UniqueName: \"kubernetes.io/projected/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-kube-api-access-mcmrc\") pod \"console-846bf86cb7-ktvjs\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") " pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:16.094399 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:16.094365    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:16.837015 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:16.836989    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"]
Jul 29 19:18:16.844516 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:16.844437    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b31b5d7_12fd_4b25_8876_430d3d6cf34b.slice/crio-01238935d4c9b52d964fa2d5b516d1b3e7e2393cc08dbbd6fc901436902ea3a4 WatchSource:0}: Error finding container 01238935d4c9b52d964fa2d5b516d1b3e7e2393cc08dbbd6fc901436902ea3a4: Status 404 returned error can't find the container with id 01238935d4c9b52d964fa2d5b516d1b3e7e2393cc08dbbd6fc901436902ea3a4
Jul 29 19:18:16.848120 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:16.848052    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-846bf86cb7-ktvjs"]
Jul 29 19:18:17.147757 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.147674    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65db98dff4-pphmd"]
Jul 29 19:18:17.153920 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.153897    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.163746 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.163724    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 19:18:17.166885 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.166865    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65db98dff4-pphmd"]
Jul 29 19:18:17.196446 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.196401    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-oauth-config\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.196586 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.196480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-oauth-serving-cert\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.196655 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.196593    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mr2q\" (UniqueName: \"kubernetes.io/projected/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-kube-api-access-2mr2q\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.196655 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.196622    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-trusted-ca-bundle\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.196754 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.196662    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-config\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.196754 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.196688    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-service-ca\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.196754 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.196715    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-serving-cert\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.297627 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.297594    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mr2q\" (UniqueName: \"kubernetes.io/projected/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-kube-api-access-2mr2q\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.297864 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.297636    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-trusted-ca-bundle\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.297864 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.297676    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-config\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.297864 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.297705    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-service-ca\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.297864 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.297736    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-serving-cert\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.298849 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.298814    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-oauth-config\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.298968 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.298915    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-oauth-serving-cert\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.299256 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.299185    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-trusted-ca-bundle\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.299256 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.299241    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-config\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.299431 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.299339    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-service-ca\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.302626 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.300476    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-oauth-serving-cert\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.302626 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.302071    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-serving-cert\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.303983 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.303959    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-oauth-config\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.307761 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.307714    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mr2q\" (UniqueName: \"kubernetes.io/projected/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-kube-api-access-2mr2q\") pod \"console-65db98dff4-pphmd\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") " pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.462084 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.462056    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:17.462746 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.462647    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846bf86cb7-ktvjs" event={"ID":"fd66a754-d938-4b31-b4ba-08cd87bbb7f7","Type":"ContainerStarted","Data":"2487a6456f4a287dc37dd8b1e22e237b99735210b106c19242f7cb15422e84ce"}
Jul 29 19:18:17.466749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.466724    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bfvhx" event={"ID":"e430e745-31c3-48f0-bfaf-ce61e230f95a","Type":"ContainerStarted","Data":"240a5a0a65ee2554a28f7dd2b9f12ec65d851049f5ba1b6322aeec5f519f03fb"}
Jul 29 19:18:17.468314 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.468288    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s" event={"ID":"8b31b5d7-12fd-4b25-8876-430d3d6cf34b","Type":"ContainerStarted","Data":"01238935d4c9b52d964fa2d5b516d1b3e7e2393cc08dbbd6fc901436902ea3a4"}
Jul 29 19:18:17.475204 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.475177    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ctkzl" event={"ID":"553abb5a-4c65-47ab-ad96-bfe1e231265c","Type":"ContainerStarted","Data":"42323eaa2d37190bb8edeeb1337cef82bbc500db63f37b7721f43d5dc100beb5"}
Jul 29 19:18:17.481711 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.481685    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dbr22" event={"ID":"ec82c59a-a7ae-46f8-884d-b3ad641bdb97","Type":"ContainerStarted","Data":"0c8f3bfc5700e2fb11a977890af364ceb7e08ef4dc2348732128e5ab0357545e"}
Jul 29 19:18:17.481807 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.481735    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dbr22" event={"ID":"ec82c59a-a7ae-46f8-884d-b3ad641bdb97","Type":"ContainerStarted","Data":"3e334c122201892cd264c9490cda3eab26ad36a7659c982f1d84d543434f89e1"}
Jul 29 19:18:17.482242 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.482222    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:17.496108 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.496050    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ctkzl" podStartSLOduration=1.708940803 podStartE2EDuration="7.496017236s" totalImagesPullingTime="5.787076433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:17.494817892 +0000 UTC m=+62.990579256" watchObservedRunningTime="2026-07-29 19:18:17.496017236 +0000 UTC m=+62.991778573"
Jul 29 19:18:17.529028 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.528970    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dbr22" podStartSLOduration=2.026044928 podStartE2EDuration="7.528949783s" totalImagesPullingTime="5.502904855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:17.52628337 +0000 UTC m=+63.022044714" watchObservedRunningTime="2026-07-29 19:18:17.528949783 +0000 UTC m=+63.024711126"
Jul 29 19:18:17.666819 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:17.666719    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65db98dff4-pphmd"]
Jul 29 19:18:17.674073 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:17.674031    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1f9ad9_77f6_4fd4_99e0_b1311c236b7f.slice/crio-95b3abbaf6220631502030f505ad16c359a227b11ee9f499d7ebc4dc47003c2f WatchSource:0}: Error finding container 95b3abbaf6220631502030f505ad16c359a227b11ee9f499d7ebc4dc47003c2f: Status 404 returned error can't find the container with id 95b3abbaf6220631502030f505ad16c359a227b11ee9f499d7ebc4dc47003c2f
Jul 29 19:18:18.492478 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.492300    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s" event={"ID":"8b31b5d7-12fd-4b25-8876-430d3d6cf34b","Type":"ContainerStarted","Data":"391d2e3fcaec1d83ef94a6e9d3afba71af68c658b1b58bbeaeb151f6095dea78"}
Jul 29 19:18:18.495652 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.493741    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"
Jul 29 19:18:18.497301 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.496942    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65db98dff4-pphmd" event={"ID":"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f","Type":"ContainerStarted","Data":"95b3abbaf6220631502030f505ad16c359a227b11ee9f499d7ebc4dc47003c2f"}
Jul 29 19:18:18.499550 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.499487    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s"
Jul 29 19:18:18.523712 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.523651    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-5gs8s" podStartSLOduration=4.015133799 podStartE2EDuration="5.523633861s" totalImagesPullingTime="1.508500062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:18.522204025 +0000 UTC m=+64.017965368" watchObservedRunningTime="2026-07-29 19:18:18.523633861 +0000 UTC m=+64.019395207"
Jul 29 19:18:18.743684 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.743649    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"]
Jul 29 19:18:18.747362 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.746055    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:18.751472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.750445    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 29 19:18:18.751472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.750817    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 29 19:18:18.751472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.751044    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 19:18:18.751472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.751246    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-b4pmp\""
Jul 29 19:18:18.752160 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.752132    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 19:18:18.752361 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.752347    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 19:18:18.760127 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.760094    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"]
Jul 29 19:18:18.821299 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.820876    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/589cf6c9-8947-452f-8566-3b7358ebd3bc-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:18.821299 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.820979    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/589cf6c9-8947-452f-8566-3b7358ebd3bc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:18.821299 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.821010    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/589cf6c9-8947-452f-8566-3b7358ebd3bc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:18.821299 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.821089    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t2r9\" (UniqueName: \"kubernetes.io/projected/589cf6c9-8947-452f-8566-3b7358ebd3bc-kube-api-access-5t2r9\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:18.922953 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.921874    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/589cf6c9-8947-452f-8566-3b7358ebd3bc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:18.922953 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.921926    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/589cf6c9-8947-452f-8566-3b7358ebd3bc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:18.922953 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.921978    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t2r9\" (UniqueName: \"kubernetes.io/projected/589cf6c9-8947-452f-8566-3b7358ebd3bc-kube-api-access-5t2r9\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:18.922953 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.922044    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/589cf6c9-8947-452f-8566-3b7358ebd3bc-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:18.922953 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:18.922572    2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 29 19:18:18.922953 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:18.922642    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589cf6c9-8947-452f-8566-3b7358ebd3bc-prometheus-operator-tls podName:589cf6c9-8947-452f-8566-3b7358ebd3bc nodeName:}" failed. No retries permitted until 2026-07-29 19:18:19.422622199 +0000 UTC m=+64.918383534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/589cf6c9-8947-452f-8566-3b7358ebd3bc-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-5zqdb" (UID: "589cf6c9-8947-452f-8566-3b7358ebd3bc") : secret "prometheus-operator-tls" not found
Jul 29 19:18:18.924266 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.924202    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/589cf6c9-8947-452f-8566-3b7358ebd3bc-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:18.927546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.927505    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/589cf6c9-8947-452f-8566-3b7358ebd3bc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:18.943143 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:18.943115    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t2r9\" (UniqueName: \"kubernetes.io/projected/589cf6c9-8947-452f-8566-3b7358ebd3bc-kube-api-access-5t2r9\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:19.427543 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:19.427359    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/589cf6c9-8947-452f-8566-3b7358ebd3bc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:19.432146 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:19.432115    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/589cf6c9-8947-452f-8566-3b7358ebd3bc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-5zqdb\" (UID: \"589cf6c9-8947-452f-8566-3b7358ebd3bc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:19.663441 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:19.663231    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"
Jul 29 19:18:20.738786 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:20.738750    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:18:20.741776 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:20.741752    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 19:18:20.751358 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:20.751338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea48fa24-04c4-434f-9cb9-eb941d6fff89-original-pull-secret\") pod \"global-pull-secret-syncer-ppw48\" (UID: \"ea48fa24-04c4-434f-9cb9-eb941d6fff89\") " pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:18:20.819702 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:20.819670    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ppw48"
Jul 29 19:18:20.839841 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:20.839814    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:18:20.843112 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:20.843085    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 19:18:20.852957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:20.852938    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6fd118c-eea4-4850-a41d-849aa67b7448-metrics-certs\") pod \"network-metrics-daemon-cmpgk\" (UID: \"f6fd118c-eea4-4850-a41d-849aa67b7448\") " pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:18:20.940832 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:20.940800    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfk9\" (UniqueName: \"kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9\") pod \"network-check-target-m7mvb\" (UID: \"2db48639-ee08-485b-874c-c64954b418dc\") " pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:18:20.944616 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:20.944589    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 19:18:20.954199 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:20.954178    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 19:18:20.964373 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:20.964350    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qfk9\" (UniqueName: \"kubernetes.io/projected/2db48639-ee08-485b-874c-c64954b418dc-kube-api-access-5qfk9\") pod \"network-check-target-m7mvb\" (UID: \"2db48639-ee08-485b-874c-c64954b418dc\") " pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:18:21.112570 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:21.112494    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qxkk6\""
Jul 29 19:18:21.119896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:21.119864    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmpgk"
Jul 29 19:18:21.129380 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:21.129357    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b2qz7\""
Jul 29 19:18:21.137682 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:21.137632    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:18:22.932642 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:22.932613    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cmpgk"]
Jul 29 19:18:22.935545 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:22.935512    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6fd118c_eea4_4850_a41d_849aa67b7448.slice/crio-5acb004e36cb992fd41010ea60d1efea735716f258383d5714ea9d1ef7d36ea0 WatchSource:0}: Error finding container 5acb004e36cb992fd41010ea60d1efea735716f258383d5714ea9d1ef7d36ea0: Status 404 returned error can't find the container with id 5acb004e36cb992fd41010ea60d1efea735716f258383d5714ea9d1ef7d36ea0
Jul 29 19:18:22.935662 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:22.935622    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-5zqdb"]
Jul 29 19:18:22.938302 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:22.938280    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589cf6c9_8947_452f_8566_3b7358ebd3bc.slice/crio-43893c229705042197017497ce5dd2b9cc7b4a4d3fb16e23197bf6e6d428bc06 WatchSource:0}: Error finding container 43893c229705042197017497ce5dd2b9cc7b4a4d3fb16e23197bf6e6d428bc06: Status 404 returned error can't find the container with id 43893c229705042197017497ce5dd2b9cc7b4a4d3fb16e23197bf6e6d428bc06
Jul 29 19:18:22.968250 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:22.968216    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ppw48"]
Jul 29 19:18:22.969325 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:22.969299    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m7mvb"]
Jul 29 19:18:22.972060 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:22.972035    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea48fa24_04c4_434f_9cb9_eb941d6fff89.slice/crio-3a2cdba57309feeb7d27f74907607485ae16a94c5b7882eb7f5756b656ed7f6f WatchSource:0}: Error finding container 3a2cdba57309feeb7d27f74907607485ae16a94c5b7882eb7f5756b656ed7f6f: Status 404 returned error can't find the container with id 3a2cdba57309feeb7d27f74907607485ae16a94c5b7882eb7f5756b656ed7f6f
Jul 29 19:18:22.972995 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:22.972968    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db48639_ee08_485b_874c_c64954b418dc.slice/crio-33d5531c8a92642b5838c493ced8c467f3cdd2f568c615c14a14f603b8a22411 WatchSource:0}: Error finding container 33d5531c8a92642b5838c493ced8c467f3cdd2f568c615c14a14f603b8a22411: Status 404 returned error can't find the container with id 33d5531c8a92642b5838c493ced8c467f3cdd2f568c615c14a14f603b8a22411
Jul 29 19:18:23.515574 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:23.515535    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bfvhx" event={"ID":"e430e745-31c3-48f0-bfaf-ce61e230f95a","Type":"ContainerStarted","Data":"8d95bfdaa44ad5a57a046d0ee618a85a58cb4b29da3c50f868113b150948b50b"}
Jul 29 19:18:23.516629 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:23.516598    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ppw48" event={"ID":"ea48fa24-04c4-434f-9cb9-eb941d6fff89","Type":"ContainerStarted","Data":"3a2cdba57309feeb7d27f74907607485ae16a94c5b7882eb7f5756b656ed7f6f"}
Jul 29 19:18:23.517957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:23.517931    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65db98dff4-pphmd" event={"ID":"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f","Type":"ContainerStarted","Data":"72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73"}
Jul 29 19:18:23.518991 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:23.518968    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb" event={"ID":"589cf6c9-8947-452f-8566-3b7358ebd3bc","Type":"ContainerStarted","Data":"43893c229705042197017497ce5dd2b9cc7b4a4d3fb16e23197bf6e6d428bc06"}
Jul 29 19:18:23.520319 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:23.520295    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846bf86cb7-ktvjs" event={"ID":"fd66a754-d938-4b31-b4ba-08cd87bbb7f7","Type":"ContainerStarted","Data":"43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64"}
Jul 29 19:18:23.521469 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:23.521438    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m7mvb" event={"ID":"2db48639-ee08-485b-874c-c64954b418dc","Type":"ContainerStarted","Data":"33d5531c8a92642b5838c493ced8c467f3cdd2f568c615c14a14f603b8a22411"}
Jul 29 19:18:23.522594 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:23.522551    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cmpgk" event={"ID":"f6fd118c-eea4-4850-a41d-849aa67b7448","Type":"ContainerStarted","Data":"5acb004e36cb992fd41010ea60d1efea735716f258383d5714ea9d1ef7d36ea0"}
Jul 29 19:18:23.542133 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:23.542093    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65db98dff4-pphmd" podStartSLOduration=1.667256272 podStartE2EDuration="6.542080535s" totalImagesPullingTime="4.874824263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:23.540945612 +0000 UTC m=+69.036706956" watchObservedRunningTime="2026-07-29 19:18:23.542080535 +0000 UTC m=+69.037841872"
Jul 29 19:18:23.560100 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:23.560059    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-846bf86cb7-ktvjs" podStartSLOduration=2.88244329 podStartE2EDuration="8.560046485s" totalImagesPullingTime="5.677603195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:23.559421778 +0000 UTC m=+69.055183120" watchObservedRunningTime="2026-07-29 19:18:23.560046485 +0000 UTC m=+69.055807828"
Jul 29 19:18:25.540286 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:25.539505    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cmpgk" event={"ID":"f6fd118c-eea4-4850-a41d-849aa67b7448","Type":"ContainerStarted","Data":"1874352f2caa644048269a0e84e662aa2bff771f081fc5c70e1eb3b3b19f7563"}
Jul 29 19:18:25.540286 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:25.539632    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cmpgk" event={"ID":"f6fd118c-eea4-4850-a41d-849aa67b7448","Type":"ContainerStarted","Data":"ddd28b2b5938bfc3856ab735bf21af695c3d53ab1319dee6660174883ca63437"}
Jul 29 19:18:25.542780 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:25.542693    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb" event={"ID":"589cf6c9-8947-452f-8566-3b7358ebd3bc","Type":"ContainerStarted","Data":"2cffd25bec5bde8608cf4a7a361f00efb5f4249be88ba2f6f904b348423d1dfc"}
Jul 29 19:18:25.542780 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:25.542749    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb" event={"ID":"589cf6c9-8947-452f-8566-3b7358ebd3bc","Type":"ContainerStarted","Data":"488b4b55fb94e60671dc8b2fd34ef893649e543fee39733a11a97a6067563592"}
Jul 29 19:18:25.557782 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:25.556540    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cmpgk" podStartSLOduration=68.242064232 podStartE2EDuration="1m10.556520141s" totalImagesPullingTime="2.314455909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:25.555959856 +0000 UTC m=+71.051721198" watchObservedRunningTime="2026-07-29 19:18:25.556520141 +0000 UTC m=+71.052281484"
Jul 29 19:18:25.573601 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:25.573546    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-5zqdb" podStartSLOduration=5.257762795 podStartE2EDuration="7.573525334s" totalImagesPullingTime="2.315762539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:25.572016643 +0000 UTC m=+71.067778012" watchObservedRunningTime="2026-07-29 19:18:25.573525334 +0000 UTC m=+71.069286677"
Jul 29 19:18:26.095154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:26.094679    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:26.095154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:26.094748    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:26.100465 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:26.100421    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:26.552706 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:26.552678    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:27.054121 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.054079    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"]
Jul 29 19:18:27.060554 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.060533    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.063112 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.063086    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 29 19:18:27.063299 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.063280    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 29 19:18:27.063386 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.063338    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9jwvf\""
Jul 29 19:18:27.070912 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.070266    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"]
Jul 29 19:18:27.085655 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.085635    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t52rl"]
Jul 29 19:18:27.088548 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.088521    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"]
Jul 29 19:18:27.088641 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.088566    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.091035 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.091019    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.091391 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.091373    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 19:18:27.091687 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.091670    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 19:18:27.091889 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.091747    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bcfpk\""
Jul 29 19:18:27.092142 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.092119    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 19:18:27.095655 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.095635    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 29 19:18:27.095791 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.095639    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 29 19:18:27.095988 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.095973    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 29 19:18:27.095988 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.095981    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-t7qjh\""
Jul 29 19:18:27.116756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.116733    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"]
Jul 29 19:18:27.195095 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195065    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7fgf\" (UniqueName: \"kubernetes.io/projected/4aa00892-4616-4831-ac2b-de884a03cf3e-kube-api-access-m7fgf\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.195095 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195101    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3b108bf-897c-44f7-80e8-6f9159c40e11-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.195327 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195131    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-accelerators-collector-config\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.195327 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195155    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.195327 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195251    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-textfile\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.195476 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195333    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6v6b\" (UniqueName: \"kubernetes.io/projected/b3b108bf-897c-44f7-80e8-6f9159c40e11-kube-api-access-h6v6b\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.195476 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195357    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-wtmp\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.195476 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195378    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4aa00892-4616-4831-ac2b-de884a03cf3e-sys\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.195476 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195407    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.195476 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195442    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3b108bf-897c-44f7-80e8-6f9159c40e11-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.195714 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195541    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c6cd90e7-355c-4173-8b7f-8febea0aa35a-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.195714 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195654    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4aa00892-4616-4831-ac2b-de884a03cf3e-root\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.195714 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195675    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.195714 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195698    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aa00892-4616-4831-ac2b-de884a03cf3e-metrics-client-ca\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.195874 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195726    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8zzn\" (UniqueName: \"kubernetes.io/projected/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-api-access-n8zzn\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.195874 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195797    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-tls\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.195874 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3b108bf-897c-44f7-80e8-6f9159c40e11-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.195874 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195869    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.196071 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.195898    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6cd90e7-355c-4173-8b7f-8febea0aa35a-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.296317 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296280    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-textfile\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.296317 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296329    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6v6b\" (UniqueName: \"kubernetes.io/projected/b3b108bf-897c-44f7-80e8-6f9159c40e11-kube-api-access-h6v6b\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.296589 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296354    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-wtmp\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.296589 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296477    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4aa00892-4616-4831-ac2b-de884a03cf3e-sys\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.296589 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296495    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-wtmp\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.296589 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296534    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.296589 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296566    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3b108bf-897c-44f7-80e8-6f9159c40e11-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.296849 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296597    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c6cd90e7-355c-4173-8b7f-8febea0aa35a-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.296849 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296614    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4aa00892-4616-4831-ac2b-de884a03cf3e-sys\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.296849 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296631    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4aa00892-4616-4831-ac2b-de884a03cf3e-root\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.296849 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296653    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-textfile\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.296849 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296667    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.296849 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296680    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4aa00892-4616-4831-ac2b-de884a03cf3e-root\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.297132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296847    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aa00892-4616-4831-ac2b-de884a03cf3e-metrics-client-ca\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.297132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296888    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8zzn\" (UniqueName: \"kubernetes.io/projected/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-api-access-n8zzn\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.297132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.296957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-tls\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.297132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.297004    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3b108bf-897c-44f7-80e8-6f9159c40e11-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.297132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.297036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.297132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.297069    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6cd90e7-355c-4173-8b7f-8febea0aa35a-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.297132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.297128    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7fgf\" (UniqueName: \"kubernetes.io/projected/4aa00892-4616-4831-ac2b-de884a03cf3e-kube-api-access-m7fgf\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.297485 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.297153    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3b108bf-897c-44f7-80e8-6f9159c40e11-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.297485 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.297165    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c6cd90e7-355c-4173-8b7f-8febea0aa35a-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.297485 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.297183    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-accelerators-collector-config\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.297485 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.297211    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.297485 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.297279    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.297485 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:27.297297    2566 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 29 19:18:27.297485 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:27.297361    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-tls podName:c6cd90e7-355c-4173-8b7f-8febea0aa35a nodeName:}" failed. No retries permitted until 2026-07-29 19:18:27.79734044 +0000 UTC m=+73.293101766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-7g8k4" (UID: "c6cd90e7-355c-4173-8b7f-8febea0aa35a") : secret "kube-state-metrics-tls" not found
Jul 29 19:18:27.297836 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.297710    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4aa00892-4616-4831-ac2b-de884a03cf3e-metrics-client-ca\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.297895 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.297842    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6cd90e7-355c-4173-8b7f-8febea0aa35a-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.298086 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.298024    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-accelerators-collector-config\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.298272 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.298196    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b3b108bf-897c-44f7-80e8-6f9159c40e11-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.299755 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.299729    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b3b108bf-897c-44f7-80e8-6f9159c40e11-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.300393 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.300366    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.300509 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.300430    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.300509 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.300492    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4aa00892-4616-4831-ac2b-de884a03cf3e-node-exporter-tls\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.300688 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.300662    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3b108bf-897c-44f7-80e8-6f9159c40e11-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.305672 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.305596    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6v6b\" (UniqueName: \"kubernetes.io/projected/b3b108bf-897c-44f7-80e8-6f9159c40e11-kube-api-access-h6v6b\") pod \"openshift-state-metrics-b8f49f5c6-955v9\" (UID: \"b3b108bf-897c-44f7-80e8-6f9159c40e11\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.307213 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.307189    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8zzn\" (UniqueName: \"kubernetes.io/projected/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-api-access-n8zzn\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.307401 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.307380    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7fgf\" (UniqueName: \"kubernetes.io/projected/4aa00892-4616-4831-ac2b-de884a03cf3e-kube-api-access-m7fgf\") pod \"node-exporter-t52rl\" (UID: \"4aa00892-4616-4831-ac2b-de884a03cf3e\") " pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.376298 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.376269    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"
Jul 29 19:18:27.402092 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.402067    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t52rl"
Jul 29 19:18:27.463504 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.463475    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:27.463696 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.463574    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:27.468791 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.468767    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:27.555655 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.555631    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:18:27.608409 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.608340    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-846bf86cb7-ktvjs"]
Jul 29 19:18:27.802400 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.802355    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:27.805636 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:27.805609    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6cd90e7-355c-4173-8b7f-8febea0aa35a-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-7g8k4\" (UID: \"c6cd90e7-355c-4173-8b7f-8febea0aa35a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:28.007669 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.007636    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"
Jul 29 19:18:28.191521 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.191483    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:18:28.196321 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.196296    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.199052 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.198983    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 19:18:28.199052 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.198984    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dvbdr\""
Jul 29 19:18:28.199269 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.199250    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 19:18:28.199364 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.199276    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 19:18:28.199364 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.199338    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 19:18:28.199561 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.199545    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 19:18:28.199561 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.199554    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 19:18:28.199705 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.199550    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 19:18:28.199834 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.199812    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 19:18:28.199834 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.199825    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 19:18:28.213079 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.212993    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:18:28.305763 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.305677    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.305763 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.305727    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.305763 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.305762    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.306020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.305784    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-web-config\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.306020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.305832    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-config-out\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.306020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.305863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.306020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.305887    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.306020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.305907    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-config-volume\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.306020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.305931    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.306020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.305959    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.306020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.305991    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.306020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.306017    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.306340 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.306043    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zp8s\" (UniqueName: \"kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-kube-api-access-6zp8s\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.406942 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.406901    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.406942 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.406948    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407165 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.406970    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-config-volume\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407256 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407320 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407290    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407375 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407330    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407375 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407504 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407397    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zp8s\" (UniqueName: \"kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-kube-api-access-6zp8s\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407504 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407622 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407500    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407622 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407553    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407622 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407582    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-web-config\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407771 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407621    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-config-out\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407824 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407776    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407912 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407882    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.407912 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:28.407884    2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 29 19:18:28.408092 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:28.407975    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-main-tls podName:beaea6f6-95c0-4880-8064-d891548dc9fa nodeName:}" failed. No retries permitted until 2026-07-29 19:18:28.907954631 +0000 UTC m=+74.403715964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa") : secret "alertmanager-main-tls" not found
Jul 29 19:18:28.408092 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.407971    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.410587 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.410541    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-config-volume\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.410587 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.410562    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.410977 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.410952    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-web-config\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.411066 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.410963    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-config-out\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.411325 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.411300    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.411431 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.411355    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.411947 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.411929    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.412248 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.412227    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.416664 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.416643    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zp8s\" (UniqueName: \"kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-kube-api-access-6zp8s\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.503753 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.503723    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dbr22"
Jul 29 19:18:28.912837 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.912803    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:28.915770 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:28.915745    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:29.107122 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.107092    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:18:29.155557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.155526    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"]
Jul 29 19:18:29.159305 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.159278    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.162884 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.162827    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1p6gfue8hool1\""
Jul 29 19:18:29.162884 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.162878    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 29 19:18:29.163049 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.162827    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 29 19:18:29.163049 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.162982    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 29 19:18:29.163136 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.163109    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 29 19:18:29.163180 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.163137    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-97l7h\""
Jul 29 19:18:29.163214 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.163187    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 29 19:18:29.174605 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.174583    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"]
Jul 29 19:18:29.317054 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.317015    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-tls\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.317238 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.317073    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-grpc-tls\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.317238 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.317098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.317238 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.317121    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ea608ef-b17b-4f97-a22e-f83b56183dad-metrics-client-ca\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.317238 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.317200    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pvwj\" (UniqueName: \"kubernetes.io/projected/0ea608ef-b17b-4f97-a22e-f83b56183dad-kube-api-access-5pvwj\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.317408 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.317258    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.317408 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.317332    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.317408 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.317387    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.418275 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.418183    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.418275 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.418244    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-tls\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.418275 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.418272    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-grpc-tls\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.418567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.418299    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.418567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.418327    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ea608ef-b17b-4f97-a22e-f83b56183dad-metrics-client-ca\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.418567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.418362    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pvwj\" (UniqueName: \"kubernetes.io/projected/0ea608ef-b17b-4f97-a22e-f83b56183dad-kube-api-access-5pvwj\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.418567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.418409    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.418567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.418474    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.419805 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.419748    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ea608ef-b17b-4f97-a22e-f83b56183dad-metrics-client-ca\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.421514 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.421493    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.421615 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.421555    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.421714 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.421693    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-tls\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.421958 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.421933    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.422338 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.422318    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.423004 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.422980    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0ea608ef-b17b-4f97-a22e-f83b56183dad-secret-grpc-tls\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.427680 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.427656    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pvwj\" (UniqueName: \"kubernetes.io/projected/0ea608ef-b17b-4f97-a22e-f83b56183dad-kube-api-access-5pvwj\") pod \"thanos-querier-5fc54567b8-6vgsr\" (UID: \"0ea608ef-b17b-4f97-a22e-f83b56183dad\") " pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:29.470150 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:29.470118    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:30.267269 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.267235    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9"]
Jul 29 19:18:30.272087 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:30.272007    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3b108bf_897c_44f7_80e8_6f9159c40e11.slice/crio-b858998a0a17d519e95b2ef5bf190673b520a7626480787086f8691ab559736b WatchSource:0}: Error finding container b858998a0a17d519e95b2ef5bf190673b520a7626480787086f8691ab559736b: Status 404 returned error can't find the container with id b858998a0a17d519e95b2ef5bf190673b520a7626480787086f8691ab559736b
Jul 29 19:18:30.495839 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.495787    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4"]
Jul 29 19:18:30.498803 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:30.498772    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6cd90e7_355c_4173_8b7f_8febea0aa35a.slice/crio-85734bd6fa8096b9c90572952aa35ee55e7a272ae46d50a33c6542dce23dba75 WatchSource:0}: Error finding container 85734bd6fa8096b9c90572952aa35ee55e7a272ae46d50a33c6542dce23dba75: Status 404 returned error can't find the container with id 85734bd6fa8096b9c90572952aa35ee55e7a272ae46d50a33c6542dce23dba75
Jul 29 19:18:30.499211 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.499184    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"]
Jul 29 19:18:30.500287 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:30.500256    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea608ef_b17b_4f97_a22e_f83b56183dad.slice/crio-bb7c5a3a19998df75ad1261d5b2c54f6651417dc0e778e9d8418124e035971a5 WatchSource:0}: Error finding container bb7c5a3a19998df75ad1261d5b2c54f6651417dc0e778e9d8418124e035971a5: Status 404 returned error can't find the container with id bb7c5a3a19998df75ad1261d5b2c54f6651417dc0e778e9d8418124e035971a5
Jul 29 19:18:30.508029 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.508007    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:18:30.510713 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:30.510691    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeaea6f6_95c0_4880_8064_d891548dc9fa.slice/crio-8e548258dc8f686de885d976cdf7d3e5b16ab9c666268da6a00c7d35ba45d295 WatchSource:0}: Error finding container 8e548258dc8f686de885d976cdf7d3e5b16ab9c666268da6a00c7d35ba45d295: Status 404 returned error can't find the container with id 8e548258dc8f686de885d976cdf7d3e5b16ab9c666268da6a00c7d35ba45d295
Jul 29 19:18:30.564570 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.564500    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr" event={"ID":"0ea608ef-b17b-4f97-a22e-f83b56183dad","Type":"ContainerStarted","Data":"bb7c5a3a19998df75ad1261d5b2c54f6651417dc0e778e9d8418124e035971a5"}
Jul 29 19:18:30.565565 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.565545    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerStarted","Data":"8e548258dc8f686de885d976cdf7d3e5b16ab9c666268da6a00c7d35ba45d295"}
Jul 29 19:18:30.566830 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.566811    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m7mvb" event={"ID":"2db48639-ee08-485b-874c-c64954b418dc","Type":"ContainerStarted","Data":"3cb08439b5ca832779db8b7396463e9dac7c00b8e12abdef8c37216dcffb9061"}
Jul 29 19:18:30.566923 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.566894    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:18:30.568663 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.568644    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9" event={"ID":"b3b108bf-897c-44f7-80e8-6f9159c40e11","Type":"ContainerStarted","Data":"f6e5a82a8a370582b8fecfea292edc0dd50ee0f1622b018f0f5fb77d41ba90ad"}
Jul 29 19:18:30.568739 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.568681    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9" event={"ID":"b3b108bf-897c-44f7-80e8-6f9159c40e11","Type":"ContainerStarted","Data":"b983990587072480bcf7d942edd35c5e5e68fc02da195d4f6a9a1ddeb55522a3"}
Jul 29 19:18:30.568739 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.568715    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9" event={"ID":"b3b108bf-897c-44f7-80e8-6f9159c40e11","Type":"ContainerStarted","Data":"b858998a0a17d519e95b2ef5bf190673b520a7626480787086f8691ab559736b"}
Jul 29 19:18:30.570471 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.570427    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bfvhx" event={"ID":"e430e745-31c3-48f0-bfaf-ce61e230f95a","Type":"ContainerStarted","Data":"d9bb49010779e2534212d2074c2675f18abde1668003206a5b2a811f3646c2dc"}
Jul 29 19:18:30.571825 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.571803    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ppw48" event={"ID":"ea48fa24-04c4-434f-9cb9-eb941d6fff89","Type":"ContainerStarted","Data":"1f00db9a5cdf7f7993b4a64ca6c8a40599759698b03c3e2cd55429feea5fef8d"}
Jul 29 19:18:30.572828 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.572802    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4" event={"ID":"c6cd90e7-355c-4173-8b7f-8febea0aa35a","Type":"ContainerStarted","Data":"85734bd6fa8096b9c90572952aa35ee55e7a272ae46d50a33c6542dce23dba75"}
Jul 29 19:18:30.573780 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.573762    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t52rl" event={"ID":"4aa00892-4616-4831-ac2b-de884a03cf3e","Type":"ContainerStarted","Data":"38a825e5024ec021531f58bc1055421ed20a710d09aed5f602ad767da7122785"}
Jul 29 19:18:30.585191 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.585156    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m7mvb" podStartSLOduration=68.503755261 podStartE2EDuration="1m15.585144555s" totalImagesPullingTime="7.081389294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:30.584990459 +0000 UTC m=+76.080751804" watchObservedRunningTime="2026-07-29 19:18:30.585144555 +0000 UTC m=+76.080905896"
Jul 29 19:18:30.600801 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.600768    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ppw48" podStartSLOduration=68.502486463 podStartE2EDuration="1m15.600756599s" totalImagesPullingTime="7.098270136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:30.60052167 +0000 UTC m=+76.096283039" watchObservedRunningTime="2026-07-29 19:18:30.600756599 +0000 UTC m=+76.096517932"
Jul 29 19:18:30.620310 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:30.620276    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bfvhx" podStartSLOduration=1.6065373360000001 podStartE2EDuration="20.620268546s" totalImagesPullingTime="19.01373121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:18:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:30.620209898 +0000 UTC m=+76.115971240" watchObservedRunningTime="2026-07-29 19:18:30.620268546 +0000 UTC m=+76.116029887"
Jul 29 19:18:31.452065 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.452027    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"]
Jul 29 19:18:31.457228 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.456600    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.460716 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.460643    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 29 19:18:31.462179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.461810    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-kp4hz\""
Jul 29 19:18:31.462179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.462060    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 29 19:18:31.462348 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.462251    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 19:18:31.462576 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.462545    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6n4n9o3vabft0\""
Jul 29 19:18:31.462706 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.462650    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 29 19:18:31.472808 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.472754    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"]
Jul 29 19:18:31.537784 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.537749    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc9fe74-1cf6-46cd-a887-ca254d560efb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.537965 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.537810    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8fc9fe74-1cf6-46cd-a887-ca254d560efb-secret-metrics-server-tls\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.537965 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.537887    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8fc9fe74-1cf6-46cd-a887-ca254d560efb-secret-metrics-server-client-certs\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.537965 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.537947    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fc9fe74-1cf6-46cd-a887-ca254d560efb-client-ca-bundle\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.538179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.537968    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8fc9fe74-1cf6-46cd-a887-ca254d560efb-metrics-server-audit-profiles\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.538179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.537988    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8fc9fe74-1cf6-46cd-a887-ca254d560efb-audit-log\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.538179 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.538023    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58nhl\" (UniqueName: \"kubernetes.io/projected/8fc9fe74-1cf6-46cd-a887-ca254d560efb-kube-api-access-58nhl\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.640416 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.639475    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fc9fe74-1cf6-46cd-a887-ca254d560efb-client-ca-bundle\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.640416 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.639526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8fc9fe74-1cf6-46cd-a887-ca254d560efb-metrics-server-audit-profiles\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.640416 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.639558    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8fc9fe74-1cf6-46cd-a887-ca254d560efb-audit-log\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.640416 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.639595    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58nhl\" (UniqueName: \"kubernetes.io/projected/8fc9fe74-1cf6-46cd-a887-ca254d560efb-kube-api-access-58nhl\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.640416 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.639659    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc9fe74-1cf6-46cd-a887-ca254d560efb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.640416 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.639697    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8fc9fe74-1cf6-46cd-a887-ca254d560efb-secret-metrics-server-tls\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.640416 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.639748    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8fc9fe74-1cf6-46cd-a887-ca254d560efb-secret-metrics-server-client-certs\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.640937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.640534    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8fc9fe74-1cf6-46cd-a887-ca254d560efb-audit-log\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.641650 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.641604    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fc9fe74-1cf6-46cd-a887-ca254d560efb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.642041 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.642017    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8fc9fe74-1cf6-46cd-a887-ca254d560efb-metrics-server-audit-profiles\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.643294 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.643272    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8fc9fe74-1cf6-46cd-a887-ca254d560efb-secret-metrics-server-tls\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.644347 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.644308    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8fc9fe74-1cf6-46cd-a887-ca254d560efb-client-ca-bundle\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.645281 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.645242    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8fc9fe74-1cf6-46cd-a887-ca254d560efb-secret-metrics-server-client-certs\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.650994 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.650955    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58nhl\" (UniqueName: \"kubernetes.io/projected/8fc9fe74-1cf6-46cd-a887-ca254d560efb-kube-api-access-58nhl\") pod \"metrics-server-7ffb854bd7-7nf7v\" (UID: \"8fc9fe74-1cf6-46cd-a887-ca254d560efb\") " pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.770730 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.770678    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:31.871861 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.871828    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"]
Jul 29 19:18:31.875797 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.875694    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-654bf478d5-9cd8t"]
Jul 29 19:18:31.875926 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.875862    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"
Jul 29 19:18:31.878604 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.878576    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 29 19:18:31.878731 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.878712    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5nqkt\""
Jul 29 19:18:31.879012 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.878995    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:31.892734 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.892666    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"]
Jul 29 19:18:31.894352 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:31.894331    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654bf478d5-9cd8t"]
Jul 29 19:18:32.043943 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.043865    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-service-ca\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.043943 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.043913    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d642388-3350-4f36-a995-942581144c66-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-b66cn\" (UID: \"8d642388-3350-4f36-a995-942581144c66\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"
Jul 29 19:18:32.044164 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.044067    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz8n8\" (UniqueName: \"kubernetes.io/projected/ccf28e1a-0106-4170-97f2-38629e6893f6-kube-api-access-nz8n8\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.044164 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.044139    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-oauth-config\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.044272 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.044196    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-serving-cert\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.044272 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.044244    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-console-config\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.044376 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.044285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-oauth-serving-cert\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.044376 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.044304    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-trusted-ca-bundle\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.044497 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.044404    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fv4q\" (UniqueName: \"kubernetes.io/projected/8d642388-3350-4f36-a995-942581144c66-kube-api-access-6fv4q\") pod \"monitoring-plugin-68b7fc9574-b66cn\" (UID: \"8d642388-3350-4f36-a995-942581144c66\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"
Jul 29 19:18:32.145032 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.144997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-service-ca\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.145032 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.145031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d642388-3350-4f36-a995-942581144c66-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-b66cn\" (UID: \"8d642388-3350-4f36-a995-942581144c66\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"
Jul 29 19:18:32.145197 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.145050    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz8n8\" (UniqueName: \"kubernetes.io/projected/ccf28e1a-0106-4170-97f2-38629e6893f6-kube-api-access-nz8n8\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.145197 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.145105    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-oauth-config\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.145339 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.145303    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-serving-cert\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.145587 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.145361    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-console-config\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.145587 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.145423    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-oauth-serving-cert\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.145587 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.145471    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-trusted-ca-bundle\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.145587 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.145548    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fv4q\" (UniqueName: \"kubernetes.io/projected/8d642388-3350-4f36-a995-942581144c66-kube-api-access-6fv4q\") pod \"monitoring-plugin-68b7fc9574-b66cn\" (UID: \"8d642388-3350-4f36-a995-942581144c66\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"
Jul 29 19:18:32.145987 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.145961    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-service-ca\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.146085 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.146042    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-console-config\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.146351 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.146326    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-oauth-serving-cert\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.146533 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.146517    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-trusted-ca-bundle\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.147680 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.147657    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d642388-3350-4f36-a995-942581144c66-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-b66cn\" (UID: \"8d642388-3350-4f36-a995-942581144c66\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"
Jul 29 19:18:32.147882 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.147865    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-oauth-config\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.147944 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.147867    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-serving-cert\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.153539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.153518    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fv4q\" (UniqueName: \"kubernetes.io/projected/8d642388-3350-4f36-a995-942581144c66-kube-api-access-6fv4q\") pod \"monitoring-plugin-68b7fc9574-b66cn\" (UID: \"8d642388-3350-4f36-a995-942581144c66\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"
Jul 29 19:18:32.153831 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.153814    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz8n8\" (UniqueName: \"kubernetes.io/projected/ccf28e1a-0106-4170-97f2-38629e6893f6-kube-api-access-nz8n8\") pod \"console-654bf478d5-9cd8t\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") " pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:32.192348 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.192327    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"
Jul 29 19:18:32.199740 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:32.199714    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:33.108954 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.108906    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654bf478d5-9cd8t"]
Jul 29 19:18:33.118034 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:33.117901    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccf28e1a_0106_4170_97f2_38629e6893f6.slice/crio-dac5c77effecee6457d491dc083661830e676c8d9ec4e164505d54d6dbec60d3 WatchSource:0}: Error finding container dac5c77effecee6457d491dc083661830e676c8d9ec4e164505d54d6dbec60d3: Status 404 returned error can't find the container with id dac5c77effecee6457d491dc083661830e676c8d9ec4e164505d54d6dbec60d3
Jul 29 19:18:33.154757 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.154146    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"]
Jul 29 19:18:33.171537 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:33.169845    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d642388_3350_4f36_a995_942581144c66.slice/crio-cfea83c26203980dfea526ea23f66b8b8434c489e5e83948db14b97bb3c20ea7 WatchSource:0}: Error finding container cfea83c26203980dfea526ea23f66b8b8434c489e5e83948db14b97bb3c20ea7: Status 404 returned error can't find the container with id cfea83c26203980dfea526ea23f66b8b8434c489e5e83948db14b97bb3c20ea7
Jul 29 19:18:33.352520 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.352496    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"]
Jul 29 19:18:33.355947 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:33.355918    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc9fe74_1cf6_46cd_a887_ca254d560efb.slice/crio-0d4ddea9bf860887d7c48371aab0875cb9d61c87d31267d421c92d9781f2443a WatchSource:0}: Error finding container 0d4ddea9bf860887d7c48371aab0875cb9d61c87d31267d421c92d9781f2443a: Status 404 returned error can't find the container with id 0d4ddea9bf860887d7c48371aab0875cb9d61c87d31267d421c92d9781f2443a
Jul 29 19:18:33.448802 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.448769    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7756dbb7c9-wvsz8"
Jul 29 19:18:33.479302 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.479204    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 19:18:33.484301 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.484215    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.487515 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.487492    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 19:18:33.487674 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.487650    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 19:18:33.487782 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.487702    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 19:18:33.487782 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.487593    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 19:18:33.487782 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.487581    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 19:18:33.487930 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.487848    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-21cq6bipf3qh2\""
Jul 29 19:18:33.489120 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.489102    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 19:18:33.489245 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.489229    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 19:18:33.489478 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.489429    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 19:18:33.489610 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.489509    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 19:18:33.489718 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.489673    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-88vhl\""
Jul 29 19:18:33.489800 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.489778    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 19:18:33.489901 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.489895    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 19:18:33.490119 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.490048    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 19:18:33.493156 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.493139    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 19:18:33.508077 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.508049    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 19:18:33.509527 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.509499    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 19:18:33.593759 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.593702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4" event={"ID":"c6cd90e7-355c-4173-8b7f-8febea0aa35a","Type":"ContainerStarted","Data":"8899c0711db921ceb4823f47b4ec03944cf2323871dd42e4705c5d0c95086124"}
Jul 29 19:18:33.593930 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.593781    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4" event={"ID":"c6cd90e7-355c-4173-8b7f-8febea0aa35a","Type":"ContainerStarted","Data":"1c3630213559cb7bad4b2b3858a6a3a0d2d3d25f993d2b0e2fe26ef983e75b75"}
Jul 29 19:18:33.593930 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.593814    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4" event={"ID":"c6cd90e7-355c-4173-8b7f-8febea0aa35a","Type":"ContainerStarted","Data":"17385b540628cddf48d61ea9e7db48247b22d198936a129946c8480f44e08396"}
Jul 29 19:18:33.595697 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.595662    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn" event={"ID":"8d642388-3350-4f36-a995-942581144c66","Type":"ContainerStarted","Data":"cfea83c26203980dfea526ea23f66b8b8434c489e5e83948db14b97bb3c20ea7"}
Jul 29 19:18:33.597274 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.597245    2566 generic.go:356] "Generic (PLEG): container finished" podID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerID="f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb" exitCode=0
Jul 29 19:18:33.597376 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.597331    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerDied","Data":"f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb"}
Jul 29 19:18:33.599274 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.599239    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654bf478d5-9cd8t" event={"ID":"ccf28e1a-0106-4170-97f2-38629e6893f6","Type":"ContainerStarted","Data":"d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408"}
Jul 29 19:18:33.599380 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.599291    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654bf478d5-9cd8t" event={"ID":"ccf28e1a-0106-4170-97f2-38629e6893f6","Type":"ContainerStarted","Data":"dac5c77effecee6457d491dc083661830e676c8d9ec4e164505d54d6dbec60d3"}
Jul 29 19:18:33.601405 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.601384    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr" event={"ID":"0ea608ef-b17b-4f97-a22e-f83b56183dad","Type":"ContainerStarted","Data":"662f691bf28f05856c1d3ba56c32599f8e252a73bf57f17bf07642a17b970787"}
Jul 29 19:18:33.601508 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.601439    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr" event={"ID":"0ea608ef-b17b-4f97-a22e-f83b56183dad","Type":"ContainerStarted","Data":"6f0a60157867ca3fe1a40fb81da5cf7f18e839fd291ccc9fcc57c4901cbf1131"}
Jul 29 19:18:33.601508 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.601493    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr" event={"ID":"0ea608ef-b17b-4f97-a22e-f83b56183dad","Type":"ContainerStarted","Data":"979cb15e23bc874c943f63f255239bf3d225a22f55060672dae954aaee43a896"}
Jul 29 19:18:33.602936 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.602915    2566 generic.go:356] "Generic (PLEG): container finished" podID="4aa00892-4616-4831-ac2b-de884a03cf3e" containerID="1975538099cf3c077d9a2397d548b2cd95f7906b0f8671b059e6c81636612f9f" exitCode=0
Jul 29 19:18:33.603019 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.602991    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t52rl" event={"ID":"4aa00892-4616-4831-ac2b-de884a03cf3e","Type":"ContainerDied","Data":"1975538099cf3c077d9a2397d548b2cd95f7906b0f8671b059e6c81636612f9f"}
Jul 29 19:18:33.604141 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.604121    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v" event={"ID":"8fc9fe74-1cf6-46cd-a887-ca254d560efb","Type":"ContainerStarted","Data":"0d4ddea9bf860887d7c48371aab0875cb9d61c87d31267d421c92d9781f2443a"}
Jul 29 19:18:33.606601 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.606583    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9" event={"ID":"b3b108bf-897c-44f7-80e8-6f9159c40e11","Type":"ContainerStarted","Data":"66bc361bab89d7ae282a1bc6420a08cc52c1d2ca0d5d4ac6e82f5079bb7e9bee"}
Jul 29 19:18:33.616271 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.616236    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-7g8k4" podStartSLOduration=4.169370484 podStartE2EDuration="6.616223462s" totalImagesPullingTime="2.446852978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:33.614990965 +0000 UTC m=+79.110752333" watchObservedRunningTime="2026-07-29 19:18:33.616223462 +0000 UTC m=+79.111984833"
Jul 29 19:18:33.635610 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.635568    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-955v9" podStartSLOduration=4.093283506 podStartE2EDuration="6.635553012s" totalImagesPullingTime="2.542269506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:33.634443374 +0000 UTC m=+79.130204743" watchObservedRunningTime="2026-07-29 19:18:33.635553012 +0000 UTC m=+79.131314354"
Jul 29 19:18:33.658958 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.658911    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-654bf478d5-9cd8t" podStartSLOduration=2.658896042 podStartE2EDuration="2.658896042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:33.657802136 +0000 UTC m=+79.153563480" watchObservedRunningTime="2026-07-29 19:18:33.658896042 +0000 UTC m=+79.154657361"
Jul 29 19:18:33.662060 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662035    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662162 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662068    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662162 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662266 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662182    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662266 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662214    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662266 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662240    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662356 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662289    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-config-out\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662356 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662321    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662356 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662335    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662482 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662364    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662482 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662379    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662482 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662447    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662580 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662519    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-config\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662580 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662534    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662580 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662556    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662580 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-web-config\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662710 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662599    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chsh8\" (UniqueName: \"kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-kube-api-access-chsh8\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662710 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662641    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662710 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662691    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.662848 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.662717    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.763965 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.763942    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chsh8\" (UniqueName: \"kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-kube-api-access-chsh8\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.764062 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.763999    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.764100 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.764089    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.764140 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.764122    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.764190 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.764170    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.764232 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.764202    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.764265 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.764231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.764265 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.764255    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.764334 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.764285    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.764334 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.764319    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.764837 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.764812    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.765269 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.765242    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.765582    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.766120    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-config-out\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.766235    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.766264    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.766379    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.766411    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.766508    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.766590    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-config\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.766616    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.766675    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.766720    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-web-config\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.768500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.767362    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.769097 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.768743    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.769097 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.769032    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.769728 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.769704    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.771809 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.771568    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.772187 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.772137    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.772792 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.772770    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.773142 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.773104    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-web-config\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.773268 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.773248    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.773938 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.773916    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chsh8\" (UniqueName: \"kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-kube-api-access-chsh8\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.774218 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.774183    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-config-out\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.774505 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.774442    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.774603 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.774531    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.774603 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.774580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.775130 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.775108    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-config\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.775759 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.775733    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.775759 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.775745    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.797255 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.797228    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:33.967958 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:33.967695    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 19:18:33.972369 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:18:33.972339    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod400ac045_5de8_4ee7_8817_293a5fa4843d.slice/crio-49da75c335e4cb37e0db123d2d014ec114bf783f72683d30aeeb0bb5d3c6a5ed WatchSource:0}: Error finding container 49da75c335e4cb37e0db123d2d014ec114bf783f72683d30aeeb0bb5d3c6a5ed: Status 404 returned error can't find the container with id 49da75c335e4cb37e0db123d2d014ec114bf783f72683d30aeeb0bb5d3c6a5ed
Jul 29 19:18:34.613486 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:34.613425    2566 generic.go:356] "Generic (PLEG): container finished" podID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerID="a3eb383b6b691278244a544e26fd0e2ad319dbc8c7b5d1a927e38df77b663cfb" exitCode=0
Jul 29 19:18:34.614100 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:34.613494    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerDied","Data":"a3eb383b6b691278244a544e26fd0e2ad319dbc8c7b5d1a927e38df77b663cfb"}
Jul 29 19:18:34.614220 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:34.614161    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerStarted","Data":"49da75c335e4cb37e0db123d2d014ec114bf783f72683d30aeeb0bb5d3c6a5ed"}
Jul 29 19:18:34.616883 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:34.616838    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t52rl" event={"ID":"4aa00892-4616-4831-ac2b-de884a03cf3e","Type":"ContainerStarted","Data":"d78697856431d3997b8a353fb3c4b2f6b12a3e33491343763c4a26cc595ff489"}
Jul 29 19:18:34.617084 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:34.617045    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t52rl" event={"ID":"4aa00892-4616-4831-ac2b-de884a03cf3e","Type":"ContainerStarted","Data":"e83098e9415a96f5a4d76f4cd324e3c49e846be3d86f8914d1a1103481ef29f2"}
Jul 29 19:18:34.677127 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:34.677070    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t52rl" podStartSLOduration=4.922668418 podStartE2EDuration="7.677047174s" totalImagesPullingTime="2.631516534s" totalInitContainerRuntime="122.862222ms" isStatefulPod=true podCreationTimestamp="2026-07-29 19:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:34.67438766 +0000 UTC m=+80.170149003" watchObservedRunningTime="2026-07-29 19:18:34.677047174 +0000 UTC m=+80.172808517"
Jul 29 19:18:35.622957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:35.622909    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn" event={"ID":"8d642388-3350-4f36-a995-942581144c66","Type":"ContainerStarted","Data":"b810b8a3ddc3c4828110835a8ac81b92f76d7a17fc7756ac037d44b4e43c6a9e"}
Jul 29 19:18:35.623402 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:35.623060    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"
Jul 29 19:18:35.626281 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:35.626251    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr" event={"ID":"0ea608ef-b17b-4f97-a22e-f83b56183dad","Type":"ContainerStarted","Data":"7a700a80b8c6edeca4e417ccd63a47d17802ee06e0ddbd0792459ca52b092c0b"}
Jul 29 19:18:35.626400 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:35.626327    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr" event={"ID":"0ea608ef-b17b-4f97-a22e-f83b56183dad","Type":"ContainerStarted","Data":"e8172a29158ffeb3d1e22b5da8787453760131f8705ed405210e59a629ad1cb7"}
Jul 29 19:18:35.628802 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:35.628778    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn"
Jul 29 19:18:35.642235 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:35.642185    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-b66cn" podStartSLOduration=2.810718339 podStartE2EDuration="4.64216734s" totalImagesPullingTime="1.831449001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:35.640095905 +0000 UTC m=+81.135857240" watchObservedRunningTime="2026-07-29 19:18:35.64216734 +0000 UTC m=+81.137928683"
Jul 29 19:18:36.633069 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.632978    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v" event={"ID":"8fc9fe74-1cf6-46cd-a887-ca254d560efb","Type":"ContainerStarted","Data":"0010cd2b37f7a26c2e40e37527a82b8095d3578ab9118f8af252b93d4b41cc22"}
Jul 29 19:18:36.636485 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.636434    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerStarted","Data":"05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129"}
Jul 29 19:18:36.636588 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.636534    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerStarted","Data":"1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d"}
Jul 29 19:18:36.636588 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.636577    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerStarted","Data":"79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d"}
Jul 29 19:18:36.636690 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.636620    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerStarted","Data":"eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b"}
Jul 29 19:18:36.636690 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.636659    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerStarted","Data":"5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c"}
Jul 29 19:18:36.636769 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.636698    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerStarted","Data":"d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c"}
Jul 29 19:18:36.639331 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.639299    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr" event={"ID":"0ea608ef-b17b-4f97-a22e-f83b56183dad","Type":"ContainerStarted","Data":"fb1febd770d48edddac8cf9535fb1a1ed32d6de4cf6a29abaca7501d769bc7e0"}
Jul 29 19:18:36.639494 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.639477    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:36.656073 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.656029    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v" podStartSLOduration=3.0793645610000002 podStartE2EDuration="5.656014681s" totalImagesPullingTime="2.57665012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:36.653512449 +0000 UTC m=+82.149273812" watchObservedRunningTime="2026-07-29 19:18:36.656014681 +0000 UTC m=+82.151776023"
Jul 29 19:18:36.677956 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.677913    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr" podStartSLOduration=3.59324415 podStartE2EDuration="7.677895399s" totalImagesPullingTime="4.084651249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:36.675568361 +0000 UTC m=+82.171329729" watchObservedRunningTime="2026-07-29 19:18:36.677895399 +0000 UTC m=+82.173656742"
Jul 29 19:18:36.706882 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:36.706596    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.828684792 podStartE2EDuration="8.706576108s" totalImagesPullingTime="4.843584136s" totalInitContainerRuntime="34.30718ms" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:36.704668946 +0000 UTC m=+82.200430316" watchObservedRunningTime="2026-07-29 19:18:36.706576108 +0000 UTC m=+82.202337451"
Jul 29 19:18:39.657763 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:39.657674    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerStarted","Data":"eeb9bb313b178e78209d75ef173a874f021a14585e40c71e9ce9aa9381c4d0d1"}
Jul 29 19:18:39.657763 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:39.657747    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerStarted","Data":"804d43956281e7069cafe4f0a0ace98a718941bd7b00baab3a280d8b71c46d2b"}
Jul 29 19:18:39.658292 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:39.657785    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerStarted","Data":"983f61d70973805bcde728e12114b95275bc16ec1641569290ceb85b924eea0f"}
Jul 29 19:18:39.658292 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:39.657825    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerStarted","Data":"dd3af97689469bdf6afcad8274645e343568515e551783f623af53a6aa361541"}
Jul 29 19:18:39.658292 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:39.657863    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerStarted","Data":"4ab055842cf7f8a7438a2fe1effed35755ee6d2a4a36519be698bac15edecaeb"}
Jul 29 19:18:39.658292 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:39.657910    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerStarted","Data":"13ff7ff20098a80333ea869038bc6f7499716908f0dc8e1762bb852d5f2557fa"}
Jul 29 19:18:39.698663 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:39.698611    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.3128589760000002 podStartE2EDuration="6.698595437s" totalImagesPullingTime="4.359916283s" totalInitContainerRuntime="25.820178ms" isStatefulPod=false podCreationTimestamp="2026-07-29 19:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:18:39.696780386 +0000 UTC m=+85.192541743" watchObservedRunningTime="2026-07-29 19:18:39.698595437 +0000 UTC m=+85.194356778"
Jul 29 19:18:42.200383 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:42.200348    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:42.200799 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:42.200405    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:42.204925 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:42.204904    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:42.651513 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:42.651422    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5fc54567b8-6vgsr"
Jul 29 19:18:42.674977 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:42.674955    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:18:42.729788 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:42.729756    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65db98dff4-pphmd"]
Jul 29 19:18:43.798542 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:43.798511    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:18:51.771423 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:51.771391    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:51.771822 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:51.771443    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:18:53.577553 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:53.577512    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-846bf86cb7-ktvjs" podUID="fd66a754-d938-4b31-b4ba-08cd87bbb7f7" containerName="console" containerID="cri-o://43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64" gracePeriod=15
Jul 29 19:18:54.354230 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.354206    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:54.469140 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.469110    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-service-ca\") pod \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") "
Jul 29 19:18:54.469307 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.469152    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-config\") pod \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") "
Jul 29 19:18:54.469307 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.469190    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-oauth-config\") pod \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") "
Jul 29 19:18:54.469422 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.469307    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-serving-cert\") pod \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") "
Jul 29 19:18:54.469422 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.469336    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcmrc\" (UniqueName: \"kubernetes.io/projected/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-kube-api-access-mcmrc\") pod \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") "
Jul 29 19:18:54.469422 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.469370    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-oauth-serving-cert\") pod \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\" (UID: \"fd66a754-d938-4b31-b4ba-08cd87bbb7f7\") "
Jul 29 19:18:54.469640 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.469604    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-config" (OuterVolumeSpecName: "console-config") pod "fd66a754-d938-4b31-b4ba-08cd87bbb7f7" (UID: "fd66a754-d938-4b31-b4ba-08cd87bbb7f7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:18:54.469771 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.469611    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-service-ca" (OuterVolumeSpecName: "service-ca") pod "fd66a754-d938-4b31-b4ba-08cd87bbb7f7" (UID: "fd66a754-d938-4b31-b4ba-08cd87bbb7f7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:18:54.469824 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.469804    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fd66a754-d938-4b31-b4ba-08cd87bbb7f7" (UID: "fd66a754-d938-4b31-b4ba-08cd87bbb7f7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:18:54.471591 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.471562    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fd66a754-d938-4b31-b4ba-08cd87bbb7f7" (UID: "fd66a754-d938-4b31-b4ba-08cd87bbb7f7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:18:54.471698 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.471562    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-kube-api-access-mcmrc" (OuterVolumeSpecName: "kube-api-access-mcmrc") pod "fd66a754-d938-4b31-b4ba-08cd87bbb7f7" (UID: "fd66a754-d938-4b31-b4ba-08cd87bbb7f7"). InnerVolumeSpecName "kube-api-access-mcmrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:18:54.471698 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.471628    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fd66a754-d938-4b31-b4ba-08cd87bbb7f7" (UID: "fd66a754-d938-4b31-b4ba-08cd87bbb7f7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:18:54.570756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.570683    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-service-ca\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:18:54.570756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.570705    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:18:54.570756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.570718    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-oauth-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:18:54.570756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.570732    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-console-serving-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:18:54.570756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.570745    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcmrc\" (UniqueName: \"kubernetes.io/projected/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-kube-api-access-mcmrc\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:18:54.570756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.570757    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd66a754-d938-4b31-b4ba-08cd87bbb7f7-oauth-serving-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:18:54.724555 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.724524    2566 generic.go:356] "Generic (PLEG): container finished" podID="fd66a754-d938-4b31-b4ba-08cd87bbb7f7" containerID="43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64" exitCode=0
Jul 29 19:18:54.724996 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.724565    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846bf86cb7-ktvjs" event={"ID":"fd66a754-d938-4b31-b4ba-08cd87bbb7f7","Type":"ContainerDied","Data":"43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64"}
Jul 29 19:18:54.724996 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.724571    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-846bf86cb7-ktvjs"
Jul 29 19:18:54.724996 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.724613    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-846bf86cb7-ktvjs" event={"ID":"fd66a754-d938-4b31-b4ba-08cd87bbb7f7","Type":"ContainerDied","Data":"2487a6456f4a287dc37dd8b1e22e237b99735210b106c19242f7cb15422e84ce"}
Jul 29 19:18:54.724996 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.724658    2566 scope.go:117] "RemoveContainer" containerID="43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64"
Jul 29 19:18:54.733789 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.733770    2566 scope.go:117] "RemoveContainer" containerID="43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64"
Jul 29 19:18:54.734049 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:18:54.734030    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64\": container with ID starting with 43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64 not found: ID does not exist" containerID="43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64"
Jul 29 19:18:54.734108 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.734059    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64"} err="failed to get container status \"43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64\": rpc error: code = NotFound desc = could not find container \"43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64\": container with ID starting with 43bc8d8e8a223c4bdf2d2992eeeab3597f49a2b6895fb3029c3fb320e7aeca64 not found: ID does not exist"
Jul 29 19:18:54.746010 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.745975    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-846bf86cb7-ktvjs"]
Jul 29 19:18:54.751528 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:54.751504    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-846bf86cb7-ktvjs"]
Jul 29 19:18:55.108281 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:18:55.108242    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd66a754-d938-4b31-b4ba-08cd87bbb7f7" path="/var/lib/kubelet/pods/fd66a754-d938-4b31-b4ba-08cd87bbb7f7/volumes"
Jul 29 19:19:01.583211 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:01.583177    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m7mvb"
Jul 29 19:19:07.748618 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:07.748566    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65db98dff4-pphmd" podUID="2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" containerName="console" containerID="cri-o://72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73" gracePeriod=15
Jul 29 19:19:07.986285 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:07.986263    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:19:08.088767 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.088702    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-oauth-serving-cert\") pod \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") "
Jul 29 19:19:08.088767 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.088752    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mr2q\" (UniqueName: \"kubernetes.io/projected/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-kube-api-access-2mr2q\") pod \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") "
Jul 29 19:19:08.088959 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.088828    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-config\") pod \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") "
Jul 29 19:19:08.088959 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.088902    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-serving-cert\") pod \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") "
Jul 29 19:19:08.089070 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.088958    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-service-ca\") pod \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") "
Jul 29 19:19:08.089124 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.089080    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-oauth-config\") pod \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") "
Jul 29 19:19:08.089175 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.089143    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-trusted-ca-bundle\") pod \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\" (UID: \"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f\") "
Jul 29 19:19:08.089225 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.089201    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" (UID: "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:08.089379 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.089350    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-config" (OuterVolumeSpecName: "console-config") pod "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" (UID: "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:08.089512 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.089384    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-service-ca" (OuterVolumeSpecName: "service-ca") pod "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" (UID: "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:08.089568 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.089554    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:08.089568 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.089567    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-service-ca\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:08.089644 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.089576    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-oauth-serving-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:08.089691 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.089629    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" (UID: "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:08.091226 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.091193    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" (UID: "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:08.091404 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.091379    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" (UID: "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:08.091506 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.091430    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-kube-api-access-2mr2q" (OuterVolumeSpecName: "kube-api-access-2mr2q") pod "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" (UID: "2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f"). InnerVolumeSpecName "kube-api-access-2mr2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:19:08.190854 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.190821    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-oauth-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:08.190854 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.190853    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-trusted-ca-bundle\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:08.190854 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.190862    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mr2q\" (UniqueName: \"kubernetes.io/projected/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-kube-api-access-2mr2q\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:08.191018 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.190871    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f-console-serving-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:08.783491 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.783444    2566 generic.go:356] "Generic (PLEG): container finished" podID="2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" containerID="72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73" exitCode=0
Jul 29 19:19:08.783896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.783540    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65db98dff4-pphmd"
Jul 29 19:19:08.783896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.783541    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65db98dff4-pphmd" event={"ID":"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f","Type":"ContainerDied","Data":"72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73"}
Jul 29 19:19:08.783896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.783622    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65db98dff4-pphmd" event={"ID":"2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f","Type":"ContainerDied","Data":"95b3abbaf6220631502030f505ad16c359a227b11ee9f499d7ebc4dc47003c2f"}
Jul 29 19:19:08.783896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.783646    2566 scope.go:117] "RemoveContainer" containerID="72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73"
Jul 29 19:19:08.792074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.792058    2566 scope.go:117] "RemoveContainer" containerID="72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73"
Jul 29 19:19:08.792317 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:19:08.792301    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73\": container with ID starting with 72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73 not found: ID does not exist" containerID="72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73"
Jul 29 19:19:08.792382 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.792323    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73"} err="failed to get container status \"72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73\": rpc error: code = NotFound desc = could not find container \"72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73\": container with ID starting with 72207c15d9e1bc9f44f4df6804a7b94cdccfba88bfe3d91894fa36a307c98f73 not found: ID does not exist"
Jul 29 19:19:08.809194 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.809175    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65db98dff4-pphmd"]
Jul 29 19:19:08.810616 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:08.810599    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65db98dff4-pphmd"]
Jul 29 19:19:09.101546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:09.101474    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" path="/var/lib/kubelet/pods/2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f/volumes"
Jul 29 19:19:11.776607 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:11.776579    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:19:11.780257 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:11.780237    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7ffb854bd7-7nf7v"
Jul 29 19:19:33.797903 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:33.797868    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:33.817537 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:33.817511    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:33.902945 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:33.902919    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:47.482820 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.482778    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:19:47.483472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.483170    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="alertmanager" containerID="cri-o://d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c" gracePeriod=120
Jul 29 19:19:47.483472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.483219    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy-metric" containerID="cri-o://1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d" gracePeriod=120
Jul 29 19:19:47.483472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.483239    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy-web" containerID="cri-o://eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b" gracePeriod=120
Jul 29 19:19:47.483472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.483270    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="config-reloader" containerID="cri-o://5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c" gracePeriod=120
Jul 29 19:19:47.483472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.483318    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy" containerID="cri-o://79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d" gracePeriod=120
Jul 29 19:19:47.483472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.483368    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="prom-label-proxy" containerID="cri-o://05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129" gracePeriod=120
Jul 29 19:19:47.945668 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.945637    2566 generic.go:356] "Generic (PLEG): container finished" podID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerID="05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129" exitCode=0
Jul 29 19:19:47.945668 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.945663    2566 generic.go:356] "Generic (PLEG): container finished" podID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerID="79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d" exitCode=0
Jul 29 19:19:47.945668 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.945670    2566 generic.go:356] "Generic (PLEG): container finished" podID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerID="5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c" exitCode=0
Jul 29 19:19:47.945668 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.945676    2566 generic.go:356] "Generic (PLEG): container finished" podID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerID="d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c" exitCode=0
Jul 29 19:19:47.945920 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.945709    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerDied","Data":"05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129"}
Jul 29 19:19:47.945920 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.945773    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerDied","Data":"79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d"}
Jul 29 19:19:47.945920 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.945799    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerDied","Data":"5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c"}
Jul 29 19:19:47.945920 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:47.945823    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerDied","Data":"d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c"}
Jul 29 19:19:48.730387 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.730367    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:48.861278 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861184    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-main-tls\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861278 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861253    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-web-config\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861278 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861285    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-web\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861583 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861314    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-main-db\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861583 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861346    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-config-volume\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861583 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861373    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861583 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861415    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-cluster-tls-config\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861583 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861473    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-trusted-ca-bundle\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861583 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861501    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zp8s\" (UniqueName: \"kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-kube-api-access-6zp8s\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861583 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861531    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-config-out\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861922 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861617    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-tls-assets\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861922 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861660    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-metrics-client-ca\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861922 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861692    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy\") pod \"beaea6f6-95c0-4880-8064-d891548dc9fa\" (UID: \"beaea6f6-95c0-4880-8064-d891548dc9fa\") "
Jul 29 19:19:48.861922 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861700    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:19:48.862119 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.861961    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-main-db\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.862218 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.862186    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:48.863148 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.863115    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:48.864596 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.864496    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:48.864765 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.864723    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:48.864997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.864942    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:48.865220 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.865191    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-config-volume" (OuterVolumeSpecName: "config-volume") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:48.865538 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.865514    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:19:48.865636 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.865573    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:48.865701 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.865646    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-kube-api-access-6zp8s" (OuterVolumeSpecName: "kube-api-access-6zp8s") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "kube-api-access-6zp8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:19:48.865801 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.865782    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-config-out" (OuterVolumeSpecName: "config-out") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:19:48.869080 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.868963    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:48.875562 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.875541    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-web-config" (OuterVolumeSpecName: "web-config") pod "beaea6f6-95c0-4880-8064-d891548dc9fa" (UID: "beaea6f6-95c0-4880-8064-d891548dc9fa"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:48.952390 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.952357    2566 generic.go:356] "Generic (PLEG): container finished" podID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerID="1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d" exitCode=0
Jul 29 19:19:48.952390 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.952384    2566 generic.go:356] "Generic (PLEG): container finished" podID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerID="eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b" exitCode=0
Jul 29 19:19:48.952582 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.952460    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:48.952582 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.952465    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerDied","Data":"1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d"}
Jul 29 19:19:48.952669 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.952579    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerDied","Data":"eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b"}
Jul 29 19:19:48.952669 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.952633    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"beaea6f6-95c0-4880-8064-d891548dc9fa","Type":"ContainerDied","Data":"8e548258dc8f686de885d976cdf7d3e5b16ab9c666268da6a00c7d35ba45d295"}
Jul 29 19:19:48.952669 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.952642    2566 scope.go:117] "RemoveContainer" containerID="05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129"
Jul 29 19:19:48.960678 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.960636    2566 scope.go:117] "RemoveContainer" containerID="1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d"
Jul 29 19:19:48.962868 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962850    2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-cluster-tls-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.962961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962869    2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.962961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962880    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zp8s\" (UniqueName: \"kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-kube-api-access-6zp8s\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.962961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962889    2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/beaea6f6-95c0-4880-8064-d891548dc9fa-config-out\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.962961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962907    2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/beaea6f6-95c0-4880-8064-d891548dc9fa-tls-assets\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.962961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962915    2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/beaea6f6-95c0-4880-8064-d891548dc9fa-metrics-client-ca\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.962961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962927    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.962961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962939    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-main-tls\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.963178 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962969    2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-web-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.963178 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962979    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.963178 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962987    2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.963178 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.962996    2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/beaea6f6-95c0-4880-8064-d891548dc9fa-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:48.968313 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.968293    2566 scope.go:117] "RemoveContainer" containerID="79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d"
Jul 29 19:19:48.974617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.974603    2566 scope.go:117] "RemoveContainer" containerID="eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b"
Jul 29 19:19:48.977549 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.977527    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:19:48.981696 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.981675    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:19:48.982326 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.982313    2566 scope.go:117] "RemoveContainer" containerID="5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c"
Jul 29 19:19:48.990528 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.990498    2566 scope.go:117] "RemoveContainer" containerID="d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c"
Jul 29 19:19:48.997664 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:48.997645    2566 scope.go:117] "RemoveContainer" containerID="f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb"
Jul 29 19:19:49.004241 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.004226    2566 scope.go:117] "RemoveContainer" containerID="05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129"
Jul 29 19:19:49.004513 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:19:49.004489    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129\": container with ID starting with 05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129 not found: ID does not exist" containerID="05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129"
Jul 29 19:19:49.004598 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.004518    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129"} err="failed to get container status \"05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129\": rpc error: code = NotFound desc = could not find container \"05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129\": container with ID starting with 05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129 not found: ID does not exist"
Jul 29 19:19:49.004598 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.004536    2566 scope.go:117] "RemoveContainer" containerID="1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d"
Jul 29 19:19:49.004775 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:19:49.004759    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d\": container with ID starting with 1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d not found: ID does not exist" containerID="1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d"
Jul 29 19:19:49.004813 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.004781    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d"} err="failed to get container status \"1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d\": rpc error: code = NotFound desc = could not find container \"1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d\": container with ID starting with 1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d not found: ID does not exist"
Jul 29 19:19:49.004813 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.004798    2566 scope.go:117] "RemoveContainer" containerID="79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d"
Jul 29 19:19:49.005011 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:19:49.004989    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d\": container with ID starting with 79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d not found: ID does not exist" containerID="79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d"
Jul 29 19:19:49.005062 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.005013    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d"} err="failed to get container status \"79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d\": rpc error: code = NotFound desc = could not find container \"79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d\": container with ID starting with 79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d not found: ID does not exist"
Jul 29 19:19:49.005062 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.005027    2566 scope.go:117] "RemoveContainer" containerID="eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b"
Jul 29 19:19:49.005224 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:19:49.005207    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b\": container with ID starting with eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b not found: ID does not exist" containerID="eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b"
Jul 29 19:19:49.005258 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.005230    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b"} err="failed to get container status \"eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b\": rpc error: code = NotFound desc = could not find container \"eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b\": container with ID starting with eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b not found: ID does not exist"
Jul 29 19:19:49.005258 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.005248    2566 scope.go:117] "RemoveContainer" containerID="5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c"
Jul 29 19:19:49.005571 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:19:49.005546    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c\": container with ID starting with 5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c not found: ID does not exist" containerID="5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c"
Jul 29 19:19:49.005673 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.005578    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c"} err="failed to get container status \"5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c\": rpc error: code = NotFound desc = could not find container \"5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c\": container with ID starting with 5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c not found: ID does not exist"
Jul 29 19:19:49.005673 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.005598    2566 scope.go:117] "RemoveContainer" containerID="d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c"
Jul 29 19:19:49.005847 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:19:49.005830    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c\": container with ID starting with d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c not found: ID does not exist" containerID="d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c"
Jul 29 19:19:49.005892 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.005852    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c"} err="failed to get container status \"d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c\": rpc error: code = NotFound desc = could not find container \"d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c\": container with ID starting with d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c not found: ID does not exist"
Jul 29 19:19:49.005892 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.005865    2566 scope.go:117] "RemoveContainer" containerID="f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb"
Jul 29 19:19:49.006099 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:19:49.006082    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb\": container with ID starting with f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb not found: ID does not exist" containerID="f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb"
Jul 29 19:19:49.006145 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.006106    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb"} err="failed to get container status \"f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb\": rpc error: code = NotFound desc = could not find container \"f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb\": container with ID starting with f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb not found: ID does not exist"
Jul 29 19:19:49.006145 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.006120    2566 scope.go:117] "RemoveContainer" containerID="05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129"
Jul 29 19:19:49.006324 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.006305    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129"} err="failed to get container status \"05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129\": rpc error: code = NotFound desc = could not find container \"05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129\": container with ID starting with 05fe3444b2a67201f1eaed823a09d95dbc2aa510b26dc613237891d426be6129 not found: ID does not exist"
Jul 29 19:19:49.006360 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.006325    2566 scope.go:117] "RemoveContainer" containerID="1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d"
Jul 29 19:19:49.006534 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.006518    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d"} err="failed to get container status \"1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d\": rpc error: code = NotFound desc = could not find container \"1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d\": container with ID starting with 1c5ea86b92aa1fa4c56758b4c359254220533c40e8c89af7458e9b45e252c06d not found: ID does not exist"
Jul 29 19:19:49.006534 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.006533    2566 scope.go:117] "RemoveContainer" containerID="79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d"
Jul 29 19:19:49.006726 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.006711    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d"} err="failed to get container status \"79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d\": rpc error: code = NotFound desc = could not find container \"79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d\": container with ID starting with 79ead2ef14f49af7c103ddb3ab4def4a5a166b49d62214d8f72b3317faf9378d not found: ID does not exist"
Jul 29 19:19:49.006777 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.006728    2566 scope.go:117] "RemoveContainer" containerID="eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b"
Jul 29 19:19:49.006891 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.006876    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b"} err="failed to get container status \"eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b\": rpc error: code = NotFound desc = could not find container \"eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b\": container with ID starting with eae0f3592c77ffff67abbc2f14ed7fe01bec1eededc485324db98a0f62e4115b not found: ID does not exist"
Jul 29 19:19:49.006942 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.006891    2566 scope.go:117] "RemoveContainer" containerID="5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c"
Jul 29 19:19:49.007088 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.007068    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c"} err="failed to get container status \"5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c\": rpc error: code = NotFound desc = could not find container \"5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c\": container with ID starting with 5a78b75a0dd10e9affdeb935b1bda1c111fa872b1636c3a5efb83bb7005b2f9c not found: ID does not exist"
Jul 29 19:19:49.007128 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.007090    2566 scope.go:117] "RemoveContainer" containerID="d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c"
Jul 29 19:19:49.007323 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.007304    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c"} err="failed to get container status \"d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c\": rpc error: code = NotFound desc = could not find container \"d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c\": container with ID starting with d0a55076f00104b26295b3acc50fa962cc19b71945f775040e91cdf06000a16c not found: ID does not exist"
Jul 29 19:19:49.007395 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.007326    2566 scope.go:117] "RemoveContainer" containerID="f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb"
Jul 29 19:19:49.007555 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.007538    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb"} err="failed to get container status \"f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb\": rpc error: code = NotFound desc = could not find container \"f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb\": container with ID starting with f39e0067aaefb8d7f4995c577604426b688da59761dc3210d5e1dfdc76c0e7eb not found: ID does not exist"
Jul 29 19:19:49.010393 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010372    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:19:49.010689 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010676    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="alertmanager"
Jul 29 19:19:49.010730 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010691    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="alertmanager"
Jul 29 19:19:49.010730 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010698    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="config-reloader"
Jul 29 19:19:49.010730 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010704    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="config-reloader"
Jul 29 19:19:49.010730 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010710    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy-metric"
Jul 29 19:19:49.010730 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010715    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy-metric"
Jul 29 19:19:49.010730 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010722    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="init-config-reloader"
Jul 29 19:19:49.010730 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010728    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="init-config-reloader"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010738    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" containerName="console"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010744    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" containerName="console"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010751    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="prom-label-proxy"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010757    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="prom-label-proxy"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010763    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010768    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010775    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy-web"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010780    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy-web"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010791    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd66a754-d938-4b31-b4ba-08cd87bbb7f7" containerName="console"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010795    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd66a754-d938-4b31-b4ba-08cd87bbb7f7" containerName="console"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010840    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="prom-label-proxy"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010848    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd66a754-d938-4b31-b4ba-08cd87bbb7f7" containerName="console"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010855    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="alertmanager"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010861    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010866    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy-web"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010871    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d1f9ad9-77f6-4fd4-99e0-b1311c236b7f" containerName="console"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010876    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="kube-rbac-proxy-metric"
Jul 29 19:19:49.010951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.010882    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" containerName="config-reloader"
Jul 29 19:19:49.014371 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.014355    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.018265 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.017497    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 19:19:49.018265 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.017881    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dvbdr\""
Jul 29 19:19:49.018265 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.018103    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 19:19:49.018481 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.018388    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 19:19:49.018481 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.018441    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 19:19:49.018589 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.018486    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 19:19:49.018589 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.018555    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 19:19:49.018720 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.018702    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 19:19:49.019070 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.019051    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 19:19:49.023263 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.023246    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 19:19:49.029060 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.029040    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:19:49.100985 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.100960    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="beaea6f6-95c0-4880-8064-d891548dc9fa" path="/var/lib/kubelet/pods/beaea6f6-95c0-4880-8064-d891548dc9fa/volumes"
Jul 29 19:19:49.165281 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165206    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-web-config\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165281 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165243    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165281 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165269    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165498 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165287    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165498 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165395    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165498 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165442    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-config-volume\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165498 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165490    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-config-out\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165618 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165516    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165618 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165537    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165618 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165587    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165703 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165619    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9mht\" (UniqueName: \"kubernetes.io/projected/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-kube-api-access-t9mht\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165703 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.165703 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.165656    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.266513 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.266484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.266577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.266518    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.266577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.266573    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.266662 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.266632    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-config-volume\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.266662 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.266653    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-config-out\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.266833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.266800    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.266943 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.266856    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.266943 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.266894    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.266943 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.266923    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9mht\" (UniqueName: \"kubernetes.io/projected/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-kube-api-access-t9mht\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.267115 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.266951    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.267115 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.266979    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.267115 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.267020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-web-config\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.267115 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.267069    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.267318 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.267128    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.268127 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.268100    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.269752 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.269725    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-config-out\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.269830 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.269779    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.269901 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.269856    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.269901 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.269887    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.270160 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.270140    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.270258 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.270239    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.270341 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.270323    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-config-volume\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.270413 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.270398    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.270598 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.270581    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.271796 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.271777    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-web-config\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.280369 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.280349    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9mht\" (UniqueName: \"kubernetes.io/projected/d03a1bc6-34b8-41b1-96d4-46ebe49c9271-kube-api-access-t9mht\") pod \"alertmanager-main-0\" (UID: \"d03a1bc6-34b8-41b1-96d4-46ebe49c9271\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.324812 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.324791    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:19:49.456725 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.456692    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:19:49.460213 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:19:49.460189    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd03a1bc6_34b8_41b1_96d4_46ebe49c9271.slice/crio-9f439afad230f87f4b07f116c232aa4632ba98c290aa5950093b97caf9a1157a WatchSource:0}: Error finding container 9f439afad230f87f4b07f116c232aa4632ba98c290aa5950093b97caf9a1157a: Status 404 returned error can't find the container with id 9f439afad230f87f4b07f116c232aa4632ba98c290aa5950093b97caf9a1157a
Jul 29 19:19:49.958685 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.958646    2566 generic.go:356] "Generic (PLEG): container finished" podID="d03a1bc6-34b8-41b1-96d4-46ebe49c9271" containerID="8a5223bc1625fa4747e2b057c4d468988672df24981cd120932d3b093e591257" exitCode=0
Jul 29 19:19:49.959145 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.958694    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d03a1bc6-34b8-41b1-96d4-46ebe49c9271","Type":"ContainerDied","Data":"8a5223bc1625fa4747e2b057c4d468988672df24981cd120932d3b093e591257"}
Jul 29 19:19:49.959145 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:49.958747    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d03a1bc6-34b8-41b1-96d4-46ebe49c9271","Type":"ContainerStarted","Data":"9f439afad230f87f4b07f116c232aa4632ba98c290aa5950093b97caf9a1157a"}
Jul 29 19:19:50.966010 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:50.965976    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d03a1bc6-34b8-41b1-96d4-46ebe49c9271","Type":"ContainerStarted","Data":"228bdf228ff7330eb9992edf2e01163a3a051ca711c978d4e964e96bcfc62fc7"}
Jul 29 19:19:50.966398 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:50.966068    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d03a1bc6-34b8-41b1-96d4-46ebe49c9271","Type":"ContainerStarted","Data":"ea4ab529486c684a3d260a94fc51f72247ab53de63778f8258d40bde3f4efc58"}
Jul 29 19:19:50.966398 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:50.966096    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d03a1bc6-34b8-41b1-96d4-46ebe49c9271","Type":"ContainerStarted","Data":"d2e2360f29d323012eec0174b6264b6806a542cb7d39d7343aa0e7afe83d47e7"}
Jul 29 19:19:50.966398 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:50.966121    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d03a1bc6-34b8-41b1-96d4-46ebe49c9271","Type":"ContainerStarted","Data":"4d17e9b40b5d537d029b479f0befefd70c276e4b4734a003c9a64171aa873ec9"}
Jul 29 19:19:50.966398 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:50.966146    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d03a1bc6-34b8-41b1-96d4-46ebe49c9271","Type":"ContainerStarted","Data":"4e733b4c82a16e9a9062292fa83aa3a21bfca30482c4333e00f7b49d4e402153"}
Jul 29 19:19:50.966398 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:50.966167    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d03a1bc6-34b8-41b1-96d4-46ebe49c9271","Type":"ContainerStarted","Data":"336e03aeb45473336f9ce2969730eea1d5de4fc4e63792967c69eb1e38d0f4bc"}
Jul 29 19:19:50.994961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:50.994904    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.977192446 podStartE2EDuration="2.994888424s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.695978ms" isStatefulPod=false podCreationTimestamp="2026-07-29 19:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:19:50.993243696 +0000 UTC m=+156.489005057" watchObservedRunningTime="2026-07-29 19:19:50.994888424 +0000 UTC m=+156.490649799"
Jul 29 19:19:51.767190 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.767149    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 19:19:51.767612 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.767558    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="prometheus" containerID="cri-o://13ff7ff20098a80333ea869038bc6f7499716908f0dc8e1762bb852d5f2557fa" gracePeriod=600
Jul 29 19:19:51.767757 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.767725    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="thanos-sidecar" containerID="cri-o://dd3af97689469bdf6afcad8274645e343568515e551783f623af53a6aa361541" gracePeriod=600
Jul 29 19:19:51.767836 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.767740    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="config-reloader" containerID="cri-o://4ab055842cf7f8a7438a2fe1effed35755ee6d2a4a36519be698bac15edecaeb" gracePeriod=600
Jul 29 19:19:51.767836 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.767819    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy-web" containerID="cri-o://983f61d70973805bcde728e12114b95275bc16ec1641569290ceb85b924eea0f" gracePeriod=600
Jul 29 19:19:51.767939 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.767889    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy" containerID="cri-o://804d43956281e7069cafe4f0a0ace98a718941bd7b00baab3a280d8b71c46d2b" gracePeriod=600
Jul 29 19:19:51.767939 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.767830    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://eeb9bb313b178e78209d75ef173a874f021a14585e40c71e9ce9aa9381c4d0d1" gracePeriod=600
Jul 29 19:19:51.976141 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976107    2566 generic.go:356] "Generic (PLEG): container finished" podID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerID="eeb9bb313b178e78209d75ef173a874f021a14585e40c71e9ce9aa9381c4d0d1" exitCode=0
Jul 29 19:19:51.976141 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976138    2566 generic.go:356] "Generic (PLEG): container finished" podID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerID="804d43956281e7069cafe4f0a0ace98a718941bd7b00baab3a280d8b71c46d2b" exitCode=0
Jul 29 19:19:51.976546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976148    2566 generic.go:356] "Generic (PLEG): container finished" podID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerID="983f61d70973805bcde728e12114b95275bc16ec1641569290ceb85b924eea0f" exitCode=0
Jul 29 19:19:51.976546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976160    2566 generic.go:356] "Generic (PLEG): container finished" podID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerID="dd3af97689469bdf6afcad8274645e343568515e551783f623af53a6aa361541" exitCode=0
Jul 29 19:19:51.976546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976168    2566 generic.go:356] "Generic (PLEG): container finished" podID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerID="4ab055842cf7f8a7438a2fe1effed35755ee6d2a4a36519be698bac15edecaeb" exitCode=0
Jul 29 19:19:51.976546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976177    2566 generic.go:356] "Generic (PLEG): container finished" podID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerID="13ff7ff20098a80333ea869038bc6f7499716908f0dc8e1762bb852d5f2557fa" exitCode=0
Jul 29 19:19:51.976546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976179    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerDied","Data":"eeb9bb313b178e78209d75ef173a874f021a14585e40c71e9ce9aa9381c4d0d1"}
Jul 29 19:19:51.976546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976241    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerDied","Data":"804d43956281e7069cafe4f0a0ace98a718941bd7b00baab3a280d8b71c46d2b"}
Jul 29 19:19:51.976546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976276    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerDied","Data":"983f61d70973805bcde728e12114b95275bc16ec1641569290ceb85b924eea0f"}
Jul 29 19:19:51.976546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976300    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerDied","Data":"dd3af97689469bdf6afcad8274645e343568515e551783f623af53a6aa361541"}
Jul 29 19:19:51.976546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976325    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerDied","Data":"4ab055842cf7f8a7438a2fe1effed35755ee6d2a4a36519be698bac15edecaeb"}
Jul 29 19:19:51.976546 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:51.976355    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerDied","Data":"13ff7ff20098a80333ea869038bc6f7499716908f0dc8e1762bb852d5f2557fa"}
Jul 29 19:19:52.024115 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.023625    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:52.092573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092543    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-kube-rbac-proxy\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.092762 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092582    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.092762 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092620    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-db\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.092762 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092646    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-1\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.092762 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092680    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-tls-assets\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.092762 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092706    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-web-config\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.092762 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092734    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-config\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093103 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092768    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093103 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092794    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-kubelet-serving-ca-bundle\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093103 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092822    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-serving-certs-ca-bundle\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093103 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092872    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-metrics-client-ca\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093103 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092898    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-2\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093103 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092934    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-tls\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093103 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092969    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-metrics-client-certs\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093103 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.092999    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-thanos-prometheus-http-client-file\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093103 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093043    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-0\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093103 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093103    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chsh8\" (UniqueName: \"kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-kube-api-access-chsh8\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093119    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:52.093577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093147    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-config-out\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093189    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-trusted-ca-bundle\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093222    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-grpc-tls\") pod \"400ac045-5de8-4ee7-8817-293a5fa4843d\" (UID: \"400ac045-5de8-4ee7-8817-293a5fa4843d\") "
Jul 29 19:19:52.093577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093335    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:52.093577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093528    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:52.093577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093548    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.093577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093571    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.093958 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093759    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:19:52.094017 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.093975    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:52.094517 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.094492    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:52.095795 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.094817    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:52.095795 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.095160    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:52.096224 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.096193    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:52.096887 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.096388    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:52.096887 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.096430    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:19:52.096887 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.096597    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:52.096887 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.096832    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:19:52.097596 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.097572    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-config-out" (OuterVolumeSpecName: "config-out") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:19:52.097801 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.097775    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-config" (OuterVolumeSpecName: "config") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:52.098284 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.098263    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:52.098632 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.098607    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:52.098878 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.098854    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-kube-api-access-chsh8" (OuterVolumeSpecName: "kube-api-access-chsh8") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "kube-api-access-chsh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:19:52.099693 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.099672    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:52.107710 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.107687    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-web-config" (OuterVolumeSpecName: "web-config") pod "400ac045-5de8-4ee7-8817-293a5fa4843d" (UID: "400ac045-5de8-4ee7-8817-293a5fa4843d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:19:52.193946 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.193914    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.193946 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.193938    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chsh8\" (UniqueName: \"kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-kube-api-access-chsh8\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.193946 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.193948    2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-config-out\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.193959    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.193968    2566 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-grpc-tls\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.193977    2566 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-kube-rbac-proxy\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.193985    2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.193995    2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/400ac045-5de8-4ee7-8817-293a5fa4843d-prometheus-k8s-db\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.194002    2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/400ac045-5de8-4ee7-8817-293a5fa4843d-tls-assets\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.194030    2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-web-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.194038    2566 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.194046    2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.194054    2566 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.194062    2566 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.194071    2566 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/400ac045-5de8-4ee7-8817-293a5fa4843d-configmap-metrics-client-ca\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.194079    2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.194087    2566 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-secret-metrics-client-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.194154 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.194097    2566 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/400ac045-5de8-4ee7-8817-293a5fa4843d-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:19:52.984263 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.984224    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"400ac045-5de8-4ee7-8817-293a5fa4843d","Type":"ContainerDied","Data":"49da75c335e4cb37e0db123d2d014ec114bf783f72683d30aeeb0bb5d3c6a5ed"}
Jul 29 19:19:52.984263 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.984242    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:52.984916 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.984301    2566 scope.go:117] "RemoveContainer" containerID="eeb9bb313b178e78209d75ef173a874f021a14585e40c71e9ce9aa9381c4d0d1"
Jul 29 19:19:52.992259 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.992232    2566 scope.go:117] "RemoveContainer" containerID="804d43956281e7069cafe4f0a0ace98a718941bd7b00baab3a280d8b71c46d2b"
Jul 29 19:19:52.999317 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:52.999299    2566 scope.go:117] "RemoveContainer" containerID="983f61d70973805bcde728e12114b95275bc16ec1641569290ceb85b924eea0f"
Jul 29 19:19:53.005862 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.005845    2566 scope.go:117] "RemoveContainer" containerID="dd3af97689469bdf6afcad8274645e343568515e551783f623af53a6aa361541"
Jul 29 19:19:53.010788 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.010770    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 19:19:53.013571 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.013539    2566 scope.go:117] "RemoveContainer" containerID="4ab055842cf7f8a7438a2fe1effed35755ee6d2a4a36519be698bac15edecaeb"
Jul 29 19:19:53.015123 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.015104    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 19:19:53.021880 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.021864    2566 scope.go:117] "RemoveContainer" containerID="13ff7ff20098a80333ea869038bc6f7499716908f0dc8e1762bb852d5f2557fa"
Jul 29 19:19:53.028847 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.028831    2566 scope.go:117] "RemoveContainer" containerID="a3eb383b6b691278244a544e26fd0e2ad319dbc8c7b5d1a927e38df77b663cfb"
Jul 29 19:19:53.042381 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042361    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 19:19:53.042727 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042712    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy-web"
Jul 29 19:19:53.042769 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042730    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy-web"
Jul 29 19:19:53.042769 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042747    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="thanos-sidecar"
Jul 29 19:19:53.042769 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042753    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="thanos-sidecar"
Jul 29 19:19:53.042769 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042761    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="init-config-reloader"
Jul 29 19:19:53.042769 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042767    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="init-config-reloader"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042772    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="prometheus"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042777    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="prometheus"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042784    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy-thanos"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042789    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy-thanos"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042797    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="config-reloader"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042802    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="config-reloader"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042807    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042812    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042856    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="config-reloader"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042863    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042869    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy-thanos"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042877    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="prometheus"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042883    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="thanos-sidecar"
Jul 29 19:19:53.042913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.042889    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" containerName="kube-rbac-proxy-web"
Jul 29 19:19:53.046868 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.046849    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.049784 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.049760    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 19:19:53.049893 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.049843    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-21cq6bipf3qh2\""
Jul 29 19:19:53.049994 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.049976    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 19:19:53.050045 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.049991    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 19:19:53.050133 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.050112    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-88vhl\""
Jul 29 19:19:53.050368 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.050238    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 19:19:53.050368 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.050270    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 19:19:53.050368 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.050270    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 19:19:53.050368 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.050336    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 19:19:53.050368 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.050348    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 19:19:53.050707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.050302    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 19:19:53.050707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.050642    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 19:19:53.050868 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.050851    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 19:19:53.050965 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.050870    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 19:19:53.054300 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.054279    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 19:19:53.056335 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.056317    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 19:19:53.070316 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.070292    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103018    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c5922cd-43c2-4f12-9005-9e0793c68e9b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103072    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103244    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103281    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103310    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103339    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103394    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103428    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-config\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103477    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103507    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103541    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103537    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="400ac045-5de8-4ee7-8817-293a5fa4843d" path="/var/lib/kubelet/pods/400ac045-5de8-4ee7-8817-293a5fa4843d/volumes"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103570    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tpr8\" (UniqueName: \"kubernetes.io/projected/0c5922cd-43c2-4f12-9005-9e0793c68e9b-kube-api-access-7tpr8\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103666    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c5922cd-43c2-4f12-9005-9e0793c68e9b-config-out\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.105756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103709    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.106574 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103741    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-web-config\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.106574 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103774    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.106574 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103806    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.106574 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103868    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.106574 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.103900    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.204421 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204389    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.204421 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204426    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-config\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.204627 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204466    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.204627 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204493    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.204627 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204520    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.204627 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204544    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tpr8\" (UniqueName: \"kubernetes.io/projected/0c5922cd-43c2-4f12-9005-9e0793c68e9b-kube-api-access-7tpr8\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.204812 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204716    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c5922cd-43c2-4f12-9005-9e0793c68e9b-config-out\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.204812 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204775    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.204812 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204805    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-web-config\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204887    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204928    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204961    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.204989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.205033    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c5922cd-43c2-4f12-9005-9e0793c68e9b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.205059    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.205084    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.205150    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.205187    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.205212    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.205243    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.205263    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.205338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.205444    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206829 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.206217    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.206829 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.206277    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.207118 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.207091    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.207611 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.207584    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.207798 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.207778    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0c5922cd-43c2-4f12-9005-9e0793c68e9b-config-out\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.208729 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.208703    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.208847 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.208826    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.209148 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.209127    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-web-config\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.209243 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.209224    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-config\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.209623 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.209603    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.209701 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.209672    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.210094 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.210070    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0c5922cd-43c2-4f12-9005-9e0793c68e9b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.210206 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.210186    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.210752 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.210733    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0c5922cd-43c2-4f12-9005-9e0793c68e9b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.210828 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.210751    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.211132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.211110    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0c5922cd-43c2-4f12-9005-9e0793c68e9b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.213386 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.213365    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tpr8\" (UniqueName: \"kubernetes.io/projected/0c5922cd-43c2-4f12-9005-9e0793c68e9b-kube-api-access-7tpr8\") pod \"prometheus-k8s-0\" (UID: \"0c5922cd-43c2-4f12-9005-9e0793c68e9b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.357579 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.357549    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:19:53.494316 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.494291    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 19:19:53.495602 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:19:53.495578    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c5922cd_43c2_4f12_9005_9e0793c68e9b.slice/crio-503f670660f926cf072b447da181883dab2a70bbebfdd7c026b4158919805ac2 WatchSource:0}: Error finding container 503f670660f926cf072b447da181883dab2a70bbebfdd7c026b4158919805ac2: Status 404 returned error can't find the container with id 503f670660f926cf072b447da181883dab2a70bbebfdd7c026b4158919805ac2
Jul 29 19:19:53.990202 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.990160    2566 generic.go:356] "Generic (PLEG): container finished" podID="0c5922cd-43c2-4f12-9005-9e0793c68e9b" containerID="30e87b6d7b1ede71400f38af6d9c78a6153d80637ce066ba60fb17b8d54bfe04" exitCode=0
Jul 29 19:19:53.990656 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.990246    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0c5922cd-43c2-4f12-9005-9e0793c68e9b","Type":"ContainerDied","Data":"30e87b6d7b1ede71400f38af6d9c78a6153d80637ce066ba60fb17b8d54bfe04"}
Jul 29 19:19:53.990656 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:53.990315    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0c5922cd-43c2-4f12-9005-9e0793c68e9b","Type":"ContainerStarted","Data":"503f670660f926cf072b447da181883dab2a70bbebfdd7c026b4158919805ac2"}
Jul 29 19:19:54.997638 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:54.997607    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0c5922cd-43c2-4f12-9005-9e0793c68e9b","Type":"ContainerStarted","Data":"a68714ce5f4a9e8f284bec881eff3282d12d65070336d6127cc4d5ef2bdf5416"}
Jul 29 19:19:54.998142 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:54.997683    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0c5922cd-43c2-4f12-9005-9e0793c68e9b","Type":"ContainerStarted","Data":"37112b4620c309cf4f3be593da7d407c5c4a4dd751ea111e192284c2e0046e45"}
Jul 29 19:19:54.998142 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:54.997723    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0c5922cd-43c2-4f12-9005-9e0793c68e9b","Type":"ContainerStarted","Data":"5e8163ba57c5d2e3e9c52c8113a3e4aba98f716fe9f0640099b3ee5344b24a46"}
Jul 29 19:19:54.998142 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:54.997765    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0c5922cd-43c2-4f12-9005-9e0793c68e9b","Type":"ContainerStarted","Data":"709793e4bec8186ddbef1eb96ebfb12568e1071e9c8ae1c795850fde5f88ffbe"}
Jul 29 19:19:54.998142 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:54.997816    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0c5922cd-43c2-4f12-9005-9e0793c68e9b","Type":"ContainerStarted","Data":"bf30edba708edd6ff7f32acfe46834feca630cc61ced9fb8e14eb91eab3a1c5c"}
Jul 29 19:19:54.998142 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:54.997852    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0c5922cd-43c2-4f12-9005-9e0793c68e9b","Type":"ContainerStarted","Data":"7366d954e14a211f0e2366f897db94e18d439ea08c1732d50ffabd566fea0f99"}
Jul 29 19:19:55.029289 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:55.029245    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.009210728 podStartE2EDuration="2.029229208s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.01848ms" isStatefulPod=false podCreationTimestamp="2026-07-29 19:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:19:55.026843206 +0000 UTC m=+160.522604592" watchObservedRunningTime="2026-07-29 19:19:55.029229208 +0000 UTC m=+160.524990550"
Jul 29 19:19:56.388446 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.388416    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d958598b-hmx6j"]
Jul 29 19:19:56.391676 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.391655    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.402362 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.402339    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d958598b-hmx6j"]
Jul 29 19:19:56.433959 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.433932    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-trusted-ca-bundle\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.434065 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.433964    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-config\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.434065 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.434050    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-service-ca\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.434149 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.434094    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-serving-cert\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.434149 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.434124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-oauth-serving-cert\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.434214 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.434154    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-oauth-config\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.434214 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.434176    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68bst\" (UniqueName: \"kubernetes.io/projected/7bba49df-b2e1-412a-9571-60faf1f73bd1-kube-api-access-68bst\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.535438 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.535394    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-oauth-config\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.535607 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.535474    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68bst\" (UniqueName: \"kubernetes.io/projected/7bba49df-b2e1-412a-9571-60faf1f73bd1-kube-api-access-68bst\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.535607 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.535527    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-trusted-ca-bundle\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.535607 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.535559    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-config\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.535760 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.535621    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-service-ca\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.535760 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.535684    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-serving-cert\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.535760 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.535722    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-oauth-serving-cert\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.536417 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.536387    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-config\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.536560 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.536401    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-service-ca\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.536560 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.536494    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-oauth-serving-cert\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.536706 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.536676    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-trusted-ca-bundle\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.538013 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.537992    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-oauth-config\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.538528 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.538505    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-serving-cert\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.543617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.543598    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68bst\" (UniqueName: \"kubernetes.io/projected/7bba49df-b2e1-412a-9571-60faf1f73bd1-kube-api-access-68bst\") pod \"console-d958598b-hmx6j\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") " pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.702084 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.702042    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:19:56.856843 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:56.856813    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d958598b-hmx6j"]
Jul 29 19:19:56.859154 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:19:56.859124    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bba49df_b2e1_412a_9571_60faf1f73bd1.slice/crio-6ff113baf803ce1fbe7e9268440f7a8a5bab9a91b517aad4e5258cf39663f665 WatchSource:0}: Error finding container 6ff113baf803ce1fbe7e9268440f7a8a5bab9a91b517aad4e5258cf39663f665: Status 404 returned error can't find the container with id 6ff113baf803ce1fbe7e9268440f7a8a5bab9a91b517aad4e5258cf39663f665
Jul 29 19:19:57.008238 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:57.008146    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d958598b-hmx6j" event={"ID":"7bba49df-b2e1-412a-9571-60faf1f73bd1","Type":"ContainerStarted","Data":"9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51"}
Jul 29 19:19:57.008238 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:57.008205    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d958598b-hmx6j" event={"ID":"7bba49df-b2e1-412a-9571-60faf1f73bd1","Type":"ContainerStarted","Data":"6ff113baf803ce1fbe7e9268440f7a8a5bab9a91b517aad4e5258cf39663f665"}
Jul 29 19:19:57.029208 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:57.029148    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d958598b-hmx6j" podStartSLOduration=1.02913354 podStartE2EDuration="1.02913354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:19:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:19:57.028090939 +0000 UTC m=+162.523852281" watchObservedRunningTime="2026-07-29 19:19:57.02913354 +0000 UTC m=+162.524894883"
Jul 29 19:19:58.357938 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:19:58.357904    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:20:06.702995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:06.702903    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:20:06.702995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:06.702984    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:20:06.707609 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:06.707587    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:20:07.061668 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:07.061595    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:20:07.111275 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:07.111244    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654bf478d5-9cd8t"]
Jul 29 19:20:32.130101 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.130050    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-654bf478d5-9cd8t" podUID="ccf28e1a-0106-4170-97f2-38629e6893f6" containerName="console" containerID="cri-o://d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408" gracePeriod=15
Jul 29 19:20:32.370379 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.370355    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:20:32.462514 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.462483    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-oauth-serving-cert\") pod \"ccf28e1a-0106-4170-97f2-38629e6893f6\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") "
Jul 29 19:20:32.462704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.462525    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-console-config\") pod \"ccf28e1a-0106-4170-97f2-38629e6893f6\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") "
Jul 29 19:20:32.462704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.462545    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-trusted-ca-bundle\") pod \"ccf28e1a-0106-4170-97f2-38629e6893f6\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") "
Jul 29 19:20:32.462704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.462578    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-service-ca\") pod \"ccf28e1a-0106-4170-97f2-38629e6893f6\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") "
Jul 29 19:20:32.462704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.462595    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-oauth-config\") pod \"ccf28e1a-0106-4170-97f2-38629e6893f6\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") "
Jul 29 19:20:32.462704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.462629    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-serving-cert\") pod \"ccf28e1a-0106-4170-97f2-38629e6893f6\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") "
Jul 29 19:20:32.462704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.462688    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz8n8\" (UniqueName: \"kubernetes.io/projected/ccf28e1a-0106-4170-97f2-38629e6893f6-kube-api-access-nz8n8\") pod \"ccf28e1a-0106-4170-97f2-38629e6893f6\" (UID: \"ccf28e1a-0106-4170-97f2-38629e6893f6\") "
Jul 29 19:20:32.463106 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.462917    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ccf28e1a-0106-4170-97f2-38629e6893f6" (UID: "ccf28e1a-0106-4170-97f2-38629e6893f6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:20:32.463190 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.463155    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ccf28e1a-0106-4170-97f2-38629e6893f6" (UID: "ccf28e1a-0106-4170-97f2-38629e6893f6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:20:32.463190 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.463175    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-console-config" (OuterVolumeSpecName: "console-config") pod "ccf28e1a-0106-4170-97f2-38629e6893f6" (UID: "ccf28e1a-0106-4170-97f2-38629e6893f6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:20:32.463370 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.463167    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-service-ca" (OuterVolumeSpecName: "service-ca") pod "ccf28e1a-0106-4170-97f2-38629e6893f6" (UID: "ccf28e1a-0106-4170-97f2-38629e6893f6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:20:32.465012 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.464983    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccf28e1a-0106-4170-97f2-38629e6893f6-kube-api-access-nz8n8" (OuterVolumeSpecName: "kube-api-access-nz8n8") pod "ccf28e1a-0106-4170-97f2-38629e6893f6" (UID: "ccf28e1a-0106-4170-97f2-38629e6893f6"). InnerVolumeSpecName "kube-api-access-nz8n8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:20:32.465115 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.465019    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ccf28e1a-0106-4170-97f2-38629e6893f6" (UID: "ccf28e1a-0106-4170-97f2-38629e6893f6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:20:32.465115 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.465031    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ccf28e1a-0106-4170-97f2-38629e6893f6" (UID: "ccf28e1a-0106-4170-97f2-38629e6893f6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:20:32.564158 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.564121    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-oauth-serving-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:20:32.564158 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.564150    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-console-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:20:32.564158 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.564160    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-trusted-ca-bundle\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:20:32.564158 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.564169    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ccf28e1a-0106-4170-97f2-38629e6893f6-service-ca\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:20:32.564408 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.564179    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-oauth-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:20:32.564408 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.564190    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ccf28e1a-0106-4170-97f2-38629e6893f6-console-serving-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:20:32.564408 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:32.564199    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz8n8\" (UniqueName: \"kubernetes.io/projected/ccf28e1a-0106-4170-97f2-38629e6893f6-kube-api-access-nz8n8\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:20:33.166261 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:33.166225    2566 generic.go:356] "Generic (PLEG): container finished" podID="ccf28e1a-0106-4170-97f2-38629e6893f6" containerID="d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408" exitCode=0
Jul 29 19:20:33.166747 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:33.166267    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654bf478d5-9cd8t" event={"ID":"ccf28e1a-0106-4170-97f2-38629e6893f6","Type":"ContainerDied","Data":"d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408"}
Jul 29 19:20:33.166747 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:33.166279    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654bf478d5-9cd8t"
Jul 29 19:20:33.166747 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:33.166304    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654bf478d5-9cd8t" event={"ID":"ccf28e1a-0106-4170-97f2-38629e6893f6","Type":"ContainerDied","Data":"dac5c77effecee6457d491dc083661830e676c8d9ec4e164505d54d6dbec60d3"}
Jul 29 19:20:33.166747 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:33.166323    2566 scope.go:117] "RemoveContainer" containerID="d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408"
Jul 29 19:20:33.174499 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:33.174482    2566 scope.go:117] "RemoveContainer" containerID="d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408"
Jul 29 19:20:33.174793 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:20:33.174768    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408\": container with ID starting with d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408 not found: ID does not exist" containerID="d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408"
Jul 29 19:20:33.174853 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:33.174803    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408"} err="failed to get container status \"d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408\": rpc error: code = NotFound desc = could not find container \"d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408\": container with ID starting with d0568394cbc55c319428d8dfe05239825c67347789764d4c7fbca68ac1301408 not found: ID does not exist"
Jul 29 19:20:33.184240 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:33.184211    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654bf478d5-9cd8t"]
Jul 29 19:20:33.187519 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:33.187493    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-654bf478d5-9cd8t"]
Jul 29 19:20:35.101238 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:35.101201    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccf28e1a-0106-4170-97f2-38629e6893f6" path="/var/lib/kubelet/pods/ccf28e1a-0106-4170-97f2-38629e6893f6/volumes"
Jul 29 19:20:53.358538 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:53.358506    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:20:53.374091 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:53.374069    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:20:54.267159 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:20:54.267132    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 19:21:16.327266 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.327233    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d96749bc4-44lbb"]
Jul 29 19:21:16.327791 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.327735    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccf28e1a-0106-4170-97f2-38629e6893f6" containerName="console"
Jul 29 19:21:16.327791 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.327756    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccf28e1a-0106-4170-97f2-38629e6893f6" containerName="console"
Jul 29 19:21:16.327903 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.327840    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccf28e1a-0106-4170-97f2-38629e6893f6" containerName="console"
Jul 29 19:21:16.331511 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.331491    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.342116 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.342092    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d96749bc4-44lbb"]
Jul 29 19:21:16.449889 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.449838    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-config\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.449889 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.449899    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-service-ca\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.450106 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.449969    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-serving-cert\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.450106 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.449989    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-oauth-serving-cert\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.450106 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.450043    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-oauth-config\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.450106 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.450097    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg259\" (UniqueName: \"kubernetes.io/projected/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-kube-api-access-bg259\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.450231 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.450135    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-trusted-ca-bundle\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.550968 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.550919    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-service-ca\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.551148 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.550989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-serving-cert\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.551148 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.551008    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-oauth-serving-cert\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.551148 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.551031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-oauth-config\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.551148 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.551050    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg259\" (UniqueName: \"kubernetes.io/projected/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-kube-api-access-bg259\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.551344 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.551213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-trusted-ca-bundle\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.551344 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.551277    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-config\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.551818 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.551791    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-service-ca\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.551948 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.551791    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-oauth-serving-cert\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.551948 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.551918    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-config\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.552038 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.551947    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-trusted-ca-bundle\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.554020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.553994    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-serving-cert\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.554020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.554006    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-oauth-config\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.562590 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.562568    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg259\" (UniqueName: \"kubernetes.io/projected/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-kube-api-access-bg259\") pod \"console-6d96749bc4-44lbb\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") " pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.640697 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.640628    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:16.770190 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:16.770168    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d96749bc4-44lbb"]
Jul 29 19:21:16.772220 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:21:16.772192    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb0270d8_f0e9_4ffc_9ce5_4995d4d49f18.slice/crio-56aed548c1e35c0dde4cfd11612a766f94894c0c6b4ed5f2ebf9c28e8d28ba63 WatchSource:0}: Error finding container 56aed548c1e35c0dde4cfd11612a766f94894c0c6b4ed5f2ebf9c28e8d28ba63: Status 404 returned error can't find the container with id 56aed548c1e35c0dde4cfd11612a766f94894c0c6b4ed5f2ebf9c28e8d28ba63
Jul 29 19:21:17.349149 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:17.349116    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d96749bc4-44lbb" event={"ID":"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18","Type":"ContainerStarted","Data":"1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d"}
Jul 29 19:21:17.349550 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:17.349166    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d96749bc4-44lbb" event={"ID":"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18","Type":"ContainerStarted","Data":"56aed548c1e35c0dde4cfd11612a766f94894c0c6b4ed5f2ebf9c28e8d28ba63"}
Jul 29 19:21:17.369766 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:17.369719    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d96749bc4-44lbb" podStartSLOduration=1.369702352 podStartE2EDuration="1.369702352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:21:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:21:17.368182818 +0000 UTC m=+242.863944160" watchObservedRunningTime="2026-07-29 19:21:17.369702352 +0000 UTC m=+242.865463695"
Jul 29 19:21:26.641272 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:26.641232    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:26.641685 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:26.641293    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:26.645902 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:26.645877    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:27.392910 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:27.392883    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:21:27.450193 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:27.450147    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d958598b-hmx6j"]
Jul 29 19:21:52.474199 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.474161    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d958598b-hmx6j" podUID="7bba49df-b2e1-412a-9571-60faf1f73bd1" containerName="console" containerID="cri-o://9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51" gracePeriod=15
Jul 29 19:21:52.710905 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.710884    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:21:52.854597 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.854521    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-oauth-config\") pod \"7bba49df-b2e1-412a-9571-60faf1f73bd1\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") "
Jul 29 19:21:52.854597 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.854565    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-config\") pod \"7bba49df-b2e1-412a-9571-60faf1f73bd1\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") "
Jul 29 19:21:52.854597 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.854595    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-service-ca\") pod \"7bba49df-b2e1-412a-9571-60faf1f73bd1\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") "
Jul 29 19:21:52.854851 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.854646    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-oauth-serving-cert\") pod \"7bba49df-b2e1-412a-9571-60faf1f73bd1\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") "
Jul 29 19:21:52.854851 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.854677    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68bst\" (UniqueName: \"kubernetes.io/projected/7bba49df-b2e1-412a-9571-60faf1f73bd1-kube-api-access-68bst\") pod \"7bba49df-b2e1-412a-9571-60faf1f73bd1\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") "
Jul 29 19:21:52.854851 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.854696    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-serving-cert\") pod \"7bba49df-b2e1-412a-9571-60faf1f73bd1\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") "
Jul 29 19:21:52.854851 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.854727    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-trusted-ca-bundle\") pod \"7bba49df-b2e1-412a-9571-60faf1f73bd1\" (UID: \"7bba49df-b2e1-412a-9571-60faf1f73bd1\") "
Jul 29 19:21:52.855144 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.855100    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-service-ca" (OuterVolumeSpecName: "service-ca") pod "7bba49df-b2e1-412a-9571-60faf1f73bd1" (UID: "7bba49df-b2e1-412a-9571-60faf1f73bd1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:21:52.855144 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.855125    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-config" (OuterVolumeSpecName: "console-config") pod "7bba49df-b2e1-412a-9571-60faf1f73bd1" (UID: "7bba49df-b2e1-412a-9571-60faf1f73bd1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:21:52.855377 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.855154    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7bba49df-b2e1-412a-9571-60faf1f73bd1" (UID: "7bba49df-b2e1-412a-9571-60faf1f73bd1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:21:52.855377 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.855239    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7bba49df-b2e1-412a-9571-60faf1f73bd1" (UID: "7bba49df-b2e1-412a-9571-60faf1f73bd1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:21:52.856822 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.856793    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bba49df-b2e1-412a-9571-60faf1f73bd1-kube-api-access-68bst" (OuterVolumeSpecName: "kube-api-access-68bst") pod "7bba49df-b2e1-412a-9571-60faf1f73bd1" (UID: "7bba49df-b2e1-412a-9571-60faf1f73bd1"). InnerVolumeSpecName "kube-api-access-68bst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:21:52.856914 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.856830    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7bba49df-b2e1-412a-9571-60faf1f73bd1" (UID: "7bba49df-b2e1-412a-9571-60faf1f73bd1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:21:52.856914 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.856889    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7bba49df-b2e1-412a-9571-60faf1f73bd1" (UID: "7bba49df-b2e1-412a-9571-60faf1f73bd1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:21:52.955585 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.955552    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68bst\" (UniqueName: \"kubernetes.io/projected/7bba49df-b2e1-412a-9571-60faf1f73bd1-kube-api-access-68bst\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:21:52.955585 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.955585    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-serving-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:21:52.955585 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.955595    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-trusted-ca-bundle\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:21:52.955766 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.955604    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-oauth-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:21:52.955766 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.955613    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-console-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:21:52.955766 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.955622    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-service-ca\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:21:52.955766 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:52.955629    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7bba49df-b2e1-412a-9571-60faf1f73bd1-oauth-serving-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:21:53.496329 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:53.496298    2566 generic.go:356] "Generic (PLEG): container finished" podID="7bba49df-b2e1-412a-9571-60faf1f73bd1" containerID="9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51" exitCode=0
Jul 29 19:21:53.496788 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:53.496346    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d958598b-hmx6j"
Jul 29 19:21:53.496788 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:53.496384    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d958598b-hmx6j" event={"ID":"7bba49df-b2e1-412a-9571-60faf1f73bd1","Type":"ContainerDied","Data":"9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51"}
Jul 29 19:21:53.496788 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:53.496466    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d958598b-hmx6j" event={"ID":"7bba49df-b2e1-412a-9571-60faf1f73bd1","Type":"ContainerDied","Data":"6ff113baf803ce1fbe7e9268440f7a8a5bab9a91b517aad4e5258cf39663f665"}
Jul 29 19:21:53.496788 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:53.496500    2566 scope.go:117] "RemoveContainer" containerID="9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51"
Jul 29 19:21:53.504341 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:53.504323    2566 scope.go:117] "RemoveContainer" containerID="9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51"
Jul 29 19:21:53.504604 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:21:53.504586    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51\": container with ID starting with 9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51 not found: ID does not exist" containerID="9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51"
Jul 29 19:21:53.504665 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:53.504620    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51"} err="failed to get container status \"9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51\": rpc error: code = NotFound desc = could not find container \"9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51\": container with ID starting with 9c43f2aa66198421d2822e964636efcee8f42fd07eb16b7d153b48ae67966d51 not found: ID does not exist"
Jul 29 19:21:53.515364 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:53.515345    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d958598b-hmx6j"]
Jul 29 19:21:53.519866 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:53.519847    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d958598b-hmx6j"]
Jul 29 19:21:55.102306 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:21:55.102263    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bba49df-b2e1-412a-9571-60faf1f73bd1" path="/var/lib/kubelet/pods/7bba49df-b2e1-412a-9571-60faf1f73bd1/volumes"
Jul 29 19:22:14.911355 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:14.911325    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 19:22:26.153467 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.153427    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4g57r"]
Jul 29 19:22:26.156164 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.153938    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bba49df-b2e1-412a-9571-60faf1f73bd1" containerName="console"
Jul 29 19:22:26.156164 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.153953    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bba49df-b2e1-412a-9571-60faf1f73bd1" containerName="console"
Jul 29 19:22:26.156164 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.154034    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bba49df-b2e1-412a-9571-60faf1f73bd1" containerName="console"
Jul 29 19:22:26.157035 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.157018    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r"
Jul 29 19:22:26.161225 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.161207    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 19:22:26.161319 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.161258    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 19:22:26.161961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.161936    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-lg9wb\""
Jul 29 19:22:26.169762 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.169740    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4g57r"]
Jul 29 19:22:26.231834 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.231805    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd8e0c07-c86f-493e-9737-7b5310ea06e4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4g57r\" (UID: \"bd8e0c07-c86f-493e-9737-7b5310ea06e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r"
Jul 29 19:22:26.231966 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.231843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqpjj\" (UniqueName: \"kubernetes.io/projected/bd8e0c07-c86f-493e-9737-7b5310ea06e4-kube-api-access-jqpjj\") pod \"cert-manager-webhook-6486ffd997-4g57r\" (UID: \"bd8e0c07-c86f-493e-9737-7b5310ea06e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r"
Jul 29 19:22:26.333009 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.332982    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd8e0c07-c86f-493e-9737-7b5310ea06e4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4g57r\" (UID: \"bd8e0c07-c86f-493e-9737-7b5310ea06e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r"
Jul 29 19:22:26.333129 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.333018    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqpjj\" (UniqueName: \"kubernetes.io/projected/bd8e0c07-c86f-493e-9737-7b5310ea06e4-kube-api-access-jqpjj\") pod \"cert-manager-webhook-6486ffd997-4g57r\" (UID: \"bd8e0c07-c86f-493e-9737-7b5310ea06e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r"
Jul 29 19:22:26.354057 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.354024    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bd8e0c07-c86f-493e-9737-7b5310ea06e4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4g57r\" (UID: \"bd8e0c07-c86f-493e-9737-7b5310ea06e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r"
Jul 29 19:22:26.355266 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.355245    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqpjj\" (UniqueName: \"kubernetes.io/projected/bd8e0c07-c86f-493e-9737-7b5310ea06e4-kube-api-access-jqpjj\") pod \"cert-manager-webhook-6486ffd997-4g57r\" (UID: \"bd8e0c07-c86f-493e-9737-7b5310ea06e4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r"
Jul 29 19:22:26.475174 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.475150    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r"
Jul 29 19:22:26.602841 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.602817    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4g57r"]
Jul 29 19:22:26.605631 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:22:26.605605    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd8e0c07_c86f_493e_9737_7b5310ea06e4.slice/crio-aa60d49f9f496e9543a7cac453cd7a5727e2442679212a2d870351b04eefb56e WatchSource:0}: Error finding container aa60d49f9f496e9543a7cac453cd7a5727e2442679212a2d870351b04eefb56e: Status 404 returned error can't find the container with id aa60d49f9f496e9543a7cac453cd7a5727e2442679212a2d870351b04eefb56e
Jul 29 19:22:26.607563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.607550    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 19:22:26.623444 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:26.623409    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r" event={"ID":"bd8e0c07-c86f-493e-9737-7b5310ea06e4","Type":"ContainerStarted","Data":"aa60d49f9f496e9543a7cac453cd7a5727e2442679212a2d870351b04eefb56e"}
Jul 29 19:22:30.641768 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:30.641730    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r" event={"ID":"bd8e0c07-c86f-493e-9737-7b5310ea06e4","Type":"ContainerStarted","Data":"b7cacbe4e19a782269cc0c99ee782442e1c2ee87dc7f2039105abf6bafa6512b"}
Jul 29 19:22:30.642161 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:30.641859    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r"
Jul 29 19:22:30.662401 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:30.662355    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r" podStartSLOduration=1.5650835920000001 podStartE2EDuration="4.662341086s" totalImagesPullingTime="3.097257494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:22:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:22:30.65987818 +0000 UTC m=+316.155639524" watchObservedRunningTime="2026-07-29 19:22:30.662341086 +0000 UTC m=+316.158102426"
Jul 29 19:22:36.647418 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:36.647386    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4g57r"
Jul 29 19:22:52.162542 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.162510    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"]
Jul 29 19:22:52.167521 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.167503    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.176372 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.176348    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 19:22:52.176613 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.176599    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 19:22:52.176674 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.176643    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mq2dv\""
Jul 29 19:22:52.196699 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.196674    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 19:22:52.196830 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.196808    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 19:22:52.198439 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.198414    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"]
Jul 29 19:22:52.253297 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.253270    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4feffa6-de91-412e-b0f1-3f049537c568-webhook-cert\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.253405 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.253329    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e4feffa6-de91-412e-b0f1-3f049537c568-manifests\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.253405 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.253370    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e4feffa6-de91-412e-b0f1-3f049537c568-apiservice-cert\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.253405 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.253398    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxqj8\" (UniqueName: \"kubernetes.io/projected/e4feffa6-de91-412e-b0f1-3f049537c568-kube-api-access-lxqj8\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.253591 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.253505    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4feffa6-de91-412e-b0f1-3f049537c568-tmp\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.354481 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.354434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4feffa6-de91-412e-b0f1-3f049537c568-tmp\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.354655 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.354548    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4feffa6-de91-412e-b0f1-3f049537c568-webhook-cert\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.354655 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.354617    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e4feffa6-de91-412e-b0f1-3f049537c568-manifests\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.354776 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.354659    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e4feffa6-de91-412e-b0f1-3f049537c568-apiservice-cert\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.354776 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.354689    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxqj8\" (UniqueName: \"kubernetes.io/projected/e4feffa6-de91-412e-b0f1-3f049537c568-kube-api-access-lxqj8\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.355381 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.355317    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e4feffa6-de91-412e-b0f1-3f049537c568-manifests\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.357152 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.357122    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4feffa6-de91-412e-b0f1-3f049537c568-tmp\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.357569 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.357545    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4feffa6-de91-412e-b0f1-3f049537c568-webhook-cert\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.357674 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.357632    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e4feffa6-de91-412e-b0f1-3f049537c568-apiservice-cert\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.368463 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.368423    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxqj8\" (UniqueName: \"kubernetes.io/projected/e4feffa6-de91-412e-b0f1-3f049537c568-kube-api-access-lxqj8\") pod \"opendatahub-operator-controller-manager-667b7bc75f-dg7fb\" (UID: \"e4feffa6-de91-412e-b0f1-3f049537c568\") " pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.478275 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.478250    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:52.621524 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.621496    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"]
Jul 29 19:22:52.623805 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:22:52.623776    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4feffa6_de91_412e_b0f1_3f049537c568.slice/crio-71a005b6fe6165edd68f0b2fa9c55783c0a02943432a1de1e07e72d7b5083705 WatchSource:0}: Error finding container 71a005b6fe6165edd68f0b2fa9c55783c0a02943432a1de1e07e72d7b5083705: Status 404 returned error can't find the container with id 71a005b6fe6165edd68f0b2fa9c55783c0a02943432a1de1e07e72d7b5083705
Jul 29 19:22:52.728793 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:52.728724    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb" event={"ID":"e4feffa6-de91-412e-b0f1-3f049537c568","Type":"ContainerStarted","Data":"71a005b6fe6165edd68f0b2fa9c55783c0a02943432a1de1e07e72d7b5083705"}
Jul 29 19:22:55.744874 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:55.744834    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb" event={"ID":"e4feffa6-de91-412e-b0f1-3f049537c568","Type":"ContainerStarted","Data":"eb7ea1987e876ed404f0e21ebcf057574e2fe9bee6e043109a1485f4674ab3b4"}
Jul 29 19:22:56.750820 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:56.750786    2566 generic.go:356] "Generic (PLEG): container finished" podID="e4feffa6-de91-412e-b0f1-3f049537c568" containerID="eb7ea1987e876ed404f0e21ebcf057574e2fe9bee6e043109a1485f4674ab3b4" exitCode=0
Jul 29 19:22:56.751156 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:56.750839    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb" event={"ID":"e4feffa6-de91-412e-b0f1-3f049537c568","Type":"ContainerDied","Data":"eb7ea1987e876ed404f0e21ebcf057574e2fe9bee6e043109a1485f4674ab3b4"}
Jul 29 19:22:57.758805 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:57.758775    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb" event={"ID":"e4feffa6-de91-412e-b0f1-3f049537c568","Type":"ContainerStarted","Data":"31c975f6e4b8334f6af4cfcc2599da10bf2d03095312631f7f2f6c025022edf2"}
Jul 29 19:22:57.759217 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:57.758911    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:22:57.783961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:22:57.783914    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb" podStartSLOduration=1.47980954 podStartE2EDuration="5.783898583s" totalImagesPullingTime="3.250482711s" totalInitContainerRuntime="1.053606332s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:22:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:22:57.782250009 +0000 UTC m=+343.278011371" watchObservedRunningTime="2026-07-29 19:22:57.783898583 +0000 UTC m=+343.279659923"
Jul 29 19:23:08.765508 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:08.765419    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-667b7bc75f-dg7fb"
Jul 29 19:23:11.208786 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:11.208754    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"]
Jul 29 19:23:11.212268 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:11.212251    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:11.221085 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:11.221051    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"lws-controller-manager-848988bfdc-zqhxc\" is forbidden: User \"system:node:ip-10-0-129-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-129-107.ec2.internal' and this object" podUID="5dc699ad-8dcc-4d71-bff6-a9b356a281b3" pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:11.221085 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:11.221059    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"webhook-server-cert\" is forbidden: User \"system:node:ip-10-0-129-107.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-129-107.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" type="*v1.Secret"
Jul 29 19:23:11.221231 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:11.221076    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"lws-controller-manager-dockercfg-k9gxk\" is forbidden: User \"system:node:ip-10-0-129-107.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-129-107.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k9gxk\"" type="*v1.Secret"
Jul 29 19:23:11.228839 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:11.228807    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-107.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-129-107.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 29 19:23:11.228976 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:11.228956    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-129-107.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-129-107.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 29 19:23:11.232854 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:11.232826    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-cert\" is forbidden: User \"system:node:ip-10-0-129-107.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-129-107.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" type="*v1.Secret"
Jul 29 19:23:11.233253 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:11.233234    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"]
Jul 29 19:23:11.234460 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:11.234428    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"lws-manager-config\" is forbidden: User \"system:node:ip-10-0-129-107.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-129-107.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" type="*v1.ConfigMap"
Jul 29 19:23:11.319581 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:11.319557    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-cert\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:11.319708 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:11.319593    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-manager-config\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:11.319708 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:11.319638    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-metrics-cert\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:11.319708 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:11.319682    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxg5d\" (UniqueName: \"kubernetes.io/projected/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-kube-api-access-sxg5d\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:11.420884 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:11.420852    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-metrics-cert\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:11.421020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:11.420937    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxg5d\" (UniqueName: \"kubernetes.io/projected/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-kube-api-access-sxg5d\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:11.421068 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:11.421031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-cert\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:11.421068 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:11.421062    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-manager-config\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:12.083821 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:12.083792    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 19:23:12.093856 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:12.093826    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-cert\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:12.182275 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:12.182242    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 19:23:12.192390 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:12.192351    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-manager-config\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:12.369498 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:12.369375    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k9gxk\""
Jul 29 19:23:12.421792 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:12.421754    2566 secret.go:189] Couldn't get secret openshift-lws-operator/metrics-server-cert: failed to sync secret cache: timed out waiting for the condition
Jul 29 19:23:12.421981 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:12.421841    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-metrics-cert podName:5dc699ad-8dcc-4d71-bff6-a9b356a281b3 nodeName:}" failed. No retries permitted until 2026-07-29 19:23:12.92181856 +0000 UTC m=+358.417579905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-cert" (UniqueName: "kubernetes.io/secret/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-metrics-cert") pod "lws-controller-manager-848988bfdc-zqhxc" (UID: "5dc699ad-8dcc-4d71-bff6-a9b356a281b3") : failed to sync secret cache: timed out waiting for the condition
Jul 29 19:23:12.444652 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:12.444607    2566 projected.go:291] Couldn't get configMap openshift-lws-operator/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Jul 29 19:23:12.481388 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:12.481359    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:23:12.484868 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:12.484842    2566 projected.go:196] Error preparing data for projected volume kube-api-access-sxg5d for pod openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc: failed to sync configmap cache: timed out waiting for the condition
Jul 29 19:23:12.485015 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:23:12.484937    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-kube-api-access-sxg5d podName:5dc699ad-8dcc-4d71-bff6-a9b356a281b3 nodeName:}" failed. No retries permitted until 2026-07-29 19:23:12.984914841 +0000 UTC m=+358.480676163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sxg5d" (UniqueName: "kubernetes.io/projected/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-kube-api-access-sxg5d") pod "lws-controller-manager-848988bfdc-zqhxc" (UID: "5dc699ad-8dcc-4d71-bff6-a9b356a281b3") : failed to sync configmap cache: timed out waiting for the condition
Jul 29 19:23:12.549111 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:12.549079    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 19:23:12.696501 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:12.696471    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 19:23:12.933022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:12.932995    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-metrics-cert\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:12.935405 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:12.935383    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-metrics-cert\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:13.033867 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:13.033789    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxg5d\" (UniqueName: \"kubernetes.io/projected/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-kube-api-access-sxg5d\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:13.036324 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:13.036297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxg5d\" (UniqueName: \"kubernetes.io/projected/5dc699ad-8dcc-4d71-bff6-a9b356a281b3-kube-api-access-sxg5d\") pod \"lws-controller-manager-848988bfdc-zqhxc\" (UID: \"5dc699ad-8dcc-4d71-bff6-a9b356a281b3\") " pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:13.321325 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:13.321239    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:13.466784 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:13.466755    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"]
Jul 29 19:23:13.469291 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:23:13.469258    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc699ad_8dcc_4d71_bff6_a9b356a281b3.slice/crio-0c9b4777204d4c6b0b0582a6edcd7406cec80e549006d00015e5712144347892 WatchSource:0}: Error finding container 0c9b4777204d4c6b0b0582a6edcd7406cec80e549006d00015e5712144347892: Status 404 returned error can't find the container with id 0c9b4777204d4c6b0b0582a6edcd7406cec80e549006d00015e5712144347892
Jul 29 19:23:13.828026 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:13.827992    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc" event={"ID":"5dc699ad-8dcc-4d71-bff6-a9b356a281b3","Type":"ContainerStarted","Data":"0c9b4777204d4c6b0b0582a6edcd7406cec80e549006d00015e5712144347892"}
Jul 29 19:23:15.840515 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:15.840483    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc" event={"ID":"5dc699ad-8dcc-4d71-bff6-a9b356a281b3","Type":"ContainerStarted","Data":"e1c42c747fd87461537ce7c7cb62dae81d240d823fa5664da2f10c248f36ee9b"}
Jul 29 19:23:15.840913 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:15.840621    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:23:15.871806 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:15.871757    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc" podStartSLOduration=2.626172593 podStartE2EDuration="4.871738601s" totalImagesPullingTime="2.245566008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:23:15.870417467 +0000 UTC m=+361.366178838" watchObservedRunningTime="2026-07-29 19:23:15.871738601 +0000 UTC m=+361.367499943"
Jul 29 19:23:23.034352 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.034319    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"]
Jul 29 19:23:23.037965 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.037942    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"
Jul 29 19:23:23.045967 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.040921    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 19:23:23.051435 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.051394    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 19:23:23.051607 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.051588    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 19:23:23.051760 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.051742    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 19:23:23.052484 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.052438    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-x44c5\""
Jul 29 19:23:23.054208 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.054097    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"]
Jul 29 19:23:23.131383 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.131347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41c7a5b-74ab-449e-8867-e9fa003e722a-tls-certs\") pod \"kube-auth-proxy-7b6ffddf6-pc545\" (UID: \"b41c7a5b-74ab-449e-8867-e9fa003e722a\") " pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"
Jul 29 19:23:23.131383 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.131384    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s92zj\" (UniqueName: \"kubernetes.io/projected/b41c7a5b-74ab-449e-8867-e9fa003e722a-kube-api-access-s92zj\") pod \"kube-auth-proxy-7b6ffddf6-pc545\" (UID: \"b41c7a5b-74ab-449e-8867-e9fa003e722a\") " pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"
Jul 29 19:23:23.131623 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.131417    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b41c7a5b-74ab-449e-8867-e9fa003e722a-tmp\") pod \"kube-auth-proxy-7b6ffddf6-pc545\" (UID: \"b41c7a5b-74ab-449e-8867-e9fa003e722a\") " pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"
Jul 29 19:23:23.232979 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.232940    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41c7a5b-74ab-449e-8867-e9fa003e722a-tls-certs\") pod \"kube-auth-proxy-7b6ffddf6-pc545\" (UID: \"b41c7a5b-74ab-449e-8867-e9fa003e722a\") " pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"
Jul 29 19:23:23.233141 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.233003    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s92zj\" (UniqueName: \"kubernetes.io/projected/b41c7a5b-74ab-449e-8867-e9fa003e722a-kube-api-access-s92zj\") pod \"kube-auth-proxy-7b6ffddf6-pc545\" (UID: \"b41c7a5b-74ab-449e-8867-e9fa003e722a\") " pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"
Jul 29 19:23:23.233141 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.233058    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b41c7a5b-74ab-449e-8867-e9fa003e722a-tmp\") pod \"kube-auth-proxy-7b6ffddf6-pc545\" (UID: \"b41c7a5b-74ab-449e-8867-e9fa003e722a\") " pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"
Jul 29 19:23:23.235147 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.235120    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b41c7a5b-74ab-449e-8867-e9fa003e722a-tmp\") pod \"kube-auth-proxy-7b6ffddf6-pc545\" (UID: \"b41c7a5b-74ab-449e-8867-e9fa003e722a\") " pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"
Jul 29 19:23:23.235287 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.235270    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41c7a5b-74ab-449e-8867-e9fa003e722a-tls-certs\") pod \"kube-auth-proxy-7b6ffddf6-pc545\" (UID: \"b41c7a5b-74ab-449e-8867-e9fa003e722a\") " pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"
Jul 29 19:23:23.252747 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.252725    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s92zj\" (UniqueName: \"kubernetes.io/projected/b41c7a5b-74ab-449e-8867-e9fa003e722a-kube-api-access-s92zj\") pod \"kube-auth-proxy-7b6ffddf6-pc545\" (UID: \"b41c7a5b-74ab-449e-8867-e9fa003e722a\") " pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"
Jul 29 19:23:23.353600 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.353517    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"
Jul 29 19:23:23.500739 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.500707    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545"]
Jul 29 19:23:23.503545 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:23:23.503515    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41c7a5b_74ab_449e_8867_e9fa003e722a.slice/crio-7dfe42e2b3c46dc2a361deeca7b28be1beb989e3a0e999d737c3e352beab479d WatchSource:0}: Error finding container 7dfe42e2b3c46dc2a361deeca7b28be1beb989e3a0e999d737c3e352beab479d: Status 404 returned error can't find the container with id 7dfe42e2b3c46dc2a361deeca7b28be1beb989e3a0e999d737c3e352beab479d
Jul 29 19:23:23.881888 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:23.881836    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545" event={"ID":"b41c7a5b-74ab-449e-8867-e9fa003e722a","Type":"ContainerStarted","Data":"7dfe42e2b3c46dc2a361deeca7b28be1beb989e3a0e999d737c3e352beab479d"}
Jul 29 19:23:25.894141 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:25.894045    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545" event={"ID":"b41c7a5b-74ab-449e-8867-e9fa003e722a","Type":"ContainerStarted","Data":"d81eed2c115bdde8e8c8c1336a0719614ffe3d20328560374cd5a56668d13362"}
Jul 29 19:23:25.936881 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:25.936817    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b6ffddf6-pc545" podStartSLOduration=1.923113954 podStartE2EDuration="3.936796693s" totalImagesPullingTime="2.013682739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:23:25.933909087 +0000 UTC m=+371.429670429" watchObservedRunningTime="2026-07-29 19:23:25.936796693 +0000 UTC m=+371.432558036"
Jul 29 19:23:26.848829 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:23:26.848802    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-848988bfdc-zqhxc"
Jul 29 19:24:59.433056 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.433022    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"]
Jul 29 19:24:59.436641 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.436612    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:24:59.440322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.440300    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 29 19:24:59.440322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.440317    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-shqcg\""
Jul 29 19:24:59.440511 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.440381    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 29 19:24:59.441365 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.441347    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 19:24:59.441421 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.441404    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 19:24:59.448671 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.448651    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"]
Jul 29 19:24:59.459132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.459112    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fzstb\" (UID: \"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:24:59.459230 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.459149    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fzstb\" (UID: \"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:24:59.459292 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.459222    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbntm\" (UniqueName: \"kubernetes.io/projected/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-kube-api-access-tbntm\") pod \"kuadrant-console-plugin-66b5774985-fzstb\" (UID: \"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:24:59.560350 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.560318    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fzstb\" (UID: \"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:24:59.560522 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.560369    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbntm\" (UniqueName: \"kubernetes.io/projected/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-kube-api-access-tbntm\") pod \"kuadrant-console-plugin-66b5774985-fzstb\" (UID: \"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:24:59.560522 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.560479    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fzstb\" (UID: \"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:24:59.560601 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:24:59.560579    2566 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 29 19:24:59.560669 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:24:59.560657    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-plugin-serving-cert podName:a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf nodeName:}" failed. No retries permitted until 2026-07-29 19:25:00.060634992 +0000 UTC m=+465.556396312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-fzstb" (UID: "a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf") : secret "plugin-serving-cert" not found
Jul 29 19:24:59.561063 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.561046    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-fzstb\" (UID: \"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:24:59.619575 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:24:59.619530    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbntm\" (UniqueName: \"kubernetes.io/projected/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-kube-api-access-tbntm\") pod \"kuadrant-console-plugin-66b5774985-fzstb\" (UID: \"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:25:00.064768 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:00.064735    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fzstb\" (UID: \"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:25:00.067032 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:00.067002    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-fzstb\" (UID: \"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:25:00.345488 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:00.345395    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"
Jul 29 19:25:00.478077 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:00.478047    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb"]
Jul 29 19:25:00.481508 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:25:00.481476    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda92a83f2_dda8_4fe9_b17a_f3d3fc1e30cf.slice/crio-cec7ca2753c74cccfe6963db55bc4001337ea6e4aec9d533f6459d09c9fc4afd WatchSource:0}: Error finding container cec7ca2753c74cccfe6963db55bc4001337ea6e4aec9d533f6459d09c9fc4afd: Status 404 returned error can't find the container with id cec7ca2753c74cccfe6963db55bc4001337ea6e4aec9d533f6459d09c9fc4afd
Jul 29 19:25:01.319163 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:01.319109    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb" event={"ID":"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf","Type":"ContainerStarted","Data":"cec7ca2753c74cccfe6963db55bc4001337ea6e4aec9d533f6459d09c9fc4afd"}
Jul 29 19:25:10.442004 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:10.441970    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"]
Jul 29 19:25:10.463320 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:10.463290    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"]
Jul 29 19:25:10.463516 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:10.463428    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:25:10.467343 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:10.467289    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j5g8m\""
Jul 29 19:25:10.568806 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:10.568773    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18f8dfea-9c84-44d0-801b-2ee7763b8af4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q24zb\" (UID: \"18f8dfea-9c84-44d0-801b-2ee7763b8af4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:25:10.568985 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:10.568889    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd5c4\" (UniqueName: \"kubernetes.io/projected/18f8dfea-9c84-44d0-801b-2ee7763b8af4-kube-api-access-jd5c4\") pod \"kuadrant-operator-controller-manager-86878c769f-q24zb\" (UID: \"18f8dfea-9c84-44d0-801b-2ee7763b8af4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:25:10.669634 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:10.669597    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18f8dfea-9c84-44d0-801b-2ee7763b8af4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q24zb\" (UID: \"18f8dfea-9c84-44d0-801b-2ee7763b8af4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:25:10.669806 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:10.669700    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd5c4\" (UniqueName: \"kubernetes.io/projected/18f8dfea-9c84-44d0-801b-2ee7763b8af4-kube-api-access-jd5c4\") pod \"kuadrant-operator-controller-manager-86878c769f-q24zb\" (UID: \"18f8dfea-9c84-44d0-801b-2ee7763b8af4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:25:10.669979 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:10.669958    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18f8dfea-9c84-44d0-801b-2ee7763b8af4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q24zb\" (UID: \"18f8dfea-9c84-44d0-801b-2ee7763b8af4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:25:10.705561 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:10.705498    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd5c4\" (UniqueName: \"kubernetes.io/projected/18f8dfea-9c84-44d0-801b-2ee7763b8af4-kube-api-access-jd5c4\") pod \"kuadrant-operator-controller-manager-86878c769f-q24zb\" (UID: \"18f8dfea-9c84-44d0-801b-2ee7763b8af4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:25:10.777190 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:10.777158    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:25:21.765156 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:21.765131    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"]
Jul 29 19:25:21.770663 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:25:21.770633    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f8dfea_9c84_44d0_801b_2ee7763b8af4.slice/crio-2069ace9d0b57aa4fa3f279de28a51c127cbc5524a94bd68e797126cff05bd7f WatchSource:0}: Error finding container 2069ace9d0b57aa4fa3f279de28a51c127cbc5524a94bd68e797126cff05bd7f: Status 404 returned error can't find the container with id 2069ace9d0b57aa4fa3f279de28a51c127cbc5524a94bd68e797126cff05bd7f
Jul 29 19:25:22.433123 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:22.433085    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb" event={"ID":"18f8dfea-9c84-44d0-801b-2ee7763b8af4","Type":"ContainerStarted","Data":"2069ace9d0b57aa4fa3f279de28a51c127cbc5524a94bd68e797126cff05bd7f"}
Jul 29 19:25:22.434704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:22.434679    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb" event={"ID":"a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf","Type":"ContainerStarted","Data":"16bec150205100bd6901e9a2256188fd06eac085f256d94ad93192a195ff487b"}
Jul 29 19:25:28.467303 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:28.467270    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb" event={"ID":"18f8dfea-9c84-44d0-801b-2ee7763b8af4","Type":"ContainerStarted","Data":"fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268"}
Jul 29 19:25:28.467782 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:28.467354    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:25:28.489956 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:28.489913    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb" podStartSLOduration=12.330846916 podStartE2EDuration="18.489897013s" totalImagesPullingTime="6.159050097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:25:28.489159995 +0000 UTC m=+493.984921359" watchObservedRunningTime="2026-07-29 19:25:28.489897013 +0000 UTC m=+493.985658356"
Jul 29 19:25:28.490927 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:28.490895    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-fzstb" podStartSLOduration=8.244492772 podStartE2EDuration="29.490884915s" totalImagesPullingTime="21.246392143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:25:22.464686529 +0000 UTC m=+487.960447868" watchObservedRunningTime="2026-07-29 19:25:28.490884915 +0000 UTC m=+493.986646257"
Jul 29 19:25:39.475615 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:25:39.475575    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:26:38.190832 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.190755    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cf7d47f5c-8mlxw"]
Jul 29 19:26:38.194069 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.194048    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:38.196744 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.196713    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 19:26:38.197027 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.197009    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d9slh\""
Jul 29 19:26:38.198204 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.198189    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 19:26:38.210404 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.210372    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf7d47f5c-8mlxw"]
Jul 29 19:26:38.231472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.231423    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2xjg\" (UniqueName: \"kubernetes.io/projected/9eb4038b-49e5-4539-9c75-efd3e0473f6a-kube-api-access-c2xjg\") pod \"maas-controller-6cf7d47f5c-8mlxw\" (UID: \"9eb4038b-49e5-4539-9c75-efd3e0473f6a\") " pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:38.231619 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.231586    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9eb4038b-49e5-4539-9c75-efd3e0473f6a-webhook-cert\") pod \"maas-controller-6cf7d47f5c-8mlxw\" (UID: \"9eb4038b-49e5-4539-9c75-efd3e0473f6a\") " pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:38.332809 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.332772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9eb4038b-49e5-4539-9c75-efd3e0473f6a-webhook-cert\") pod \"maas-controller-6cf7d47f5c-8mlxw\" (UID: \"9eb4038b-49e5-4539-9c75-efd3e0473f6a\") " pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:38.332979 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.332839    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2xjg\" (UniqueName: \"kubernetes.io/projected/9eb4038b-49e5-4539-9c75-efd3e0473f6a-kube-api-access-c2xjg\") pod \"maas-controller-6cf7d47f5c-8mlxw\" (UID: \"9eb4038b-49e5-4539-9c75-efd3e0473f6a\") " pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:38.332979 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:26:38.332919    2566 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 29 19:26:38.333050 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:26:38.332995    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eb4038b-49e5-4539-9c75-efd3e0473f6a-webhook-cert podName:9eb4038b-49e5-4539-9c75-efd3e0473f6a nodeName:}" failed. No retries permitted until 2026-07-29 19:26:38.83297773 +0000 UTC m=+564.328739050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9eb4038b-49e5-4539-9c75-efd3e0473f6a-webhook-cert") pod "maas-controller-6cf7d47f5c-8mlxw" (UID: "9eb4038b-49e5-4539-9c75-efd3e0473f6a") : secret "maas-controller-webhook-cert" not found
Jul 29 19:26:38.350561 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.350533    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2xjg\" (UniqueName: \"kubernetes.io/projected/9eb4038b-49e5-4539-9c75-efd3e0473f6a-kube-api-access-c2xjg\") pod \"maas-controller-6cf7d47f5c-8mlxw\" (UID: \"9eb4038b-49e5-4539-9c75-efd3e0473f6a\") " pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:38.357507 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.357480    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65d7444bfc-k6hfg"]
Jul 29 19:26:38.361107 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.361088    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:26:38.371693 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.371670    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-k6hfg"]
Jul 29 19:26:38.433698 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.433669    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7td9\" (UniqueName: \"kubernetes.io/projected/ca2246db-ea6b-4d75-8667-0da5ece52c58-kube-api-access-l7td9\") pod \"maas-controller-65d7444bfc-k6hfg\" (UID: \"ca2246db-ea6b-4d75-8667-0da5ece52c58\") " pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:26:38.433867 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.433724    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca2246db-ea6b-4d75-8667-0da5ece52c58-webhook-cert\") pod \"maas-controller-65d7444bfc-k6hfg\" (UID: \"ca2246db-ea6b-4d75-8667-0da5ece52c58\") " pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:26:38.535132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.535046    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7td9\" (UniqueName: \"kubernetes.io/projected/ca2246db-ea6b-4d75-8667-0da5ece52c58-kube-api-access-l7td9\") pod \"maas-controller-65d7444bfc-k6hfg\" (UID: \"ca2246db-ea6b-4d75-8667-0da5ece52c58\") " pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:26:38.535132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.535097    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca2246db-ea6b-4d75-8667-0da5ece52c58-webhook-cert\") pod \"maas-controller-65d7444bfc-k6hfg\" (UID: \"ca2246db-ea6b-4d75-8667-0da5ece52c58\") " pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:26:38.537421 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.537395    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca2246db-ea6b-4d75-8667-0da5ece52c58-webhook-cert\") pod \"maas-controller-65d7444bfc-k6hfg\" (UID: \"ca2246db-ea6b-4d75-8667-0da5ece52c58\") " pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:26:38.544543 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.544516    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7td9\" (UniqueName: \"kubernetes.io/projected/ca2246db-ea6b-4d75-8667-0da5ece52c58-kube-api-access-l7td9\") pod \"maas-controller-65d7444bfc-k6hfg\" (UID: \"ca2246db-ea6b-4d75-8667-0da5ece52c58\") " pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:26:38.673261 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.673223    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:26:38.803929 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.803902    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-k6hfg"]
Jul 29 19:26:38.806127 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:26:38.806102    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca2246db_ea6b_4d75_8667_0da5ece52c58.slice/crio-04f991760c8a96b66fb1687c476720b026b2c3651b889accd026a970dd229da6 WatchSource:0}: Error finding container 04f991760c8a96b66fb1687c476720b026b2c3651b889accd026a970dd229da6: Status 404 returned error can't find the container with id 04f991760c8a96b66fb1687c476720b026b2c3651b889accd026a970dd229da6
Jul 29 19:26:38.838721 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.838690    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9eb4038b-49e5-4539-9c75-efd3e0473f6a-webhook-cert\") pod \"maas-controller-6cf7d47f5c-8mlxw\" (UID: \"9eb4038b-49e5-4539-9c75-efd3e0473f6a\") " pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:38.840986 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:38.840968    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9eb4038b-49e5-4539-9c75-efd3e0473f6a-webhook-cert\") pod \"maas-controller-6cf7d47f5c-8mlxw\" (UID: \"9eb4038b-49e5-4539-9c75-efd3e0473f6a\") " pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:39.105051 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:39.104978    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:39.238140 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:39.238114    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf7d47f5c-8mlxw"]
Jul 29 19:26:39.239939 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:26:39.239913    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb4038b_49e5_4539_9c75_efd3e0473f6a.slice/crio-5745a675f468ec367a19e9cfb50b19d9a797c45a7dfe6e718b4e9f4d3c4ab991 WatchSource:0}: Error finding container 5745a675f468ec367a19e9cfb50b19d9a797c45a7dfe6e718b4e9f4d3c4ab991: Status 404 returned error can't find the container with id 5745a675f468ec367a19e9cfb50b19d9a797c45a7dfe6e718b4e9f4d3c4ab991
Jul 29 19:26:39.794801 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:39.794704    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw" event={"ID":"9eb4038b-49e5-4539-9c75-efd3e0473f6a","Type":"ContainerStarted","Data":"5745a675f468ec367a19e9cfb50b19d9a797c45a7dfe6e718b4e9f4d3c4ab991"}
Jul 29 19:26:39.796668 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:39.796613    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d7444bfc-k6hfg" event={"ID":"ca2246db-ea6b-4d75-8667-0da5ece52c58","Type":"ContainerStarted","Data":"04f991760c8a96b66fb1687c476720b026b2c3651b889accd026a970dd229da6"}
Jul 29 19:26:41.808267 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:41.808224    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw" event={"ID":"9eb4038b-49e5-4539-9c75-efd3e0473f6a","Type":"ContainerStarted","Data":"147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545"}
Jul 29 19:26:41.808743 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:41.808282    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:41.809726 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:41.809702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d7444bfc-k6hfg" event={"ID":"ca2246db-ea6b-4d75-8667-0da5ece52c58","Type":"ContainerStarted","Data":"d76f1a648f08513df1214c0a9d4a513c7507a8ff7bf9de8d38cf8898910c0145"}
Jul 29 19:26:41.809826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:41.809789    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:26:41.828075 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:41.828028    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw" podStartSLOduration=1.783029864 podStartE2EDuration="3.828013766s" totalImagesPullingTime="2.044983902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:26:41.825902184 +0000 UTC m=+567.321663552" watchObservedRunningTime="2026-07-29 19:26:41.828013766 +0000 UTC m=+567.323775107"
Jul 29 19:26:41.845825 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:41.845766    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65d7444bfc-k6hfg" podStartSLOduration=1.373503313 podStartE2EDuration="3.845748752s" totalImagesPullingTime="2.472245439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:26:41.845544673 +0000 UTC m=+567.341306039" watchObservedRunningTime="2026-07-29 19:26:41.845748752 +0000 UTC m=+567.341510096"
Jul 29 19:26:52.816844 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:52.816813    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:52.817277 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:52.816885    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:26:52.892733 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:52.892701    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf7d47f5c-8mlxw"]
Jul 29 19:26:52.892958 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:52.892908    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw" podUID="9eb4038b-49e5-4539-9c75-efd3e0473f6a" containerName="manager" containerID="cri-o://147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545" gracePeriod=30
Jul 29 19:26:53.142502 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.142478    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:53.279844 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.279812    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9eb4038b-49e5-4539-9c75-efd3e0473f6a-webhook-cert\") pod \"9eb4038b-49e5-4539-9c75-efd3e0473f6a\" (UID: \"9eb4038b-49e5-4539-9c75-efd3e0473f6a\") "
Jul 29 19:26:53.280004 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.279855    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2xjg\" (UniqueName: \"kubernetes.io/projected/9eb4038b-49e5-4539-9c75-efd3e0473f6a-kube-api-access-c2xjg\") pod \"9eb4038b-49e5-4539-9c75-efd3e0473f6a\" (UID: \"9eb4038b-49e5-4539-9c75-efd3e0473f6a\") "
Jul 29 19:26:53.282036 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.282006    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9eb4038b-49e5-4539-9c75-efd3e0473f6a-kube-api-access-c2xjg" (OuterVolumeSpecName: "kube-api-access-c2xjg") pod "9eb4038b-49e5-4539-9c75-efd3e0473f6a" (UID: "9eb4038b-49e5-4539-9c75-efd3e0473f6a"). InnerVolumeSpecName "kube-api-access-c2xjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:26:53.282142 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.282103    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9eb4038b-49e5-4539-9c75-efd3e0473f6a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9eb4038b-49e5-4539-9c75-efd3e0473f6a" (UID: "9eb4038b-49e5-4539-9c75-efd3e0473f6a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:26:53.380725 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.380665    2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9eb4038b-49e5-4539-9c75-efd3e0473f6a-webhook-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:26:53.380725 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.380691    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2xjg\" (UniqueName: \"kubernetes.io/projected/9eb4038b-49e5-4539-9c75-efd3e0473f6a-kube-api-access-c2xjg\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:26:53.867820 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.867784    2566 generic.go:356] "Generic (PLEG): container finished" podID="9eb4038b-49e5-4539-9c75-efd3e0473f6a" containerID="147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545" exitCode=0
Jul 29 19:26:53.868261 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.867834    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw"
Jul 29 19:26:53.868261 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.867872    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw" event={"ID":"9eb4038b-49e5-4539-9c75-efd3e0473f6a","Type":"ContainerDied","Data":"147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545"}
Jul 29 19:26:53.868261 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.867938    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf7d47f5c-8mlxw" event={"ID":"9eb4038b-49e5-4539-9c75-efd3e0473f6a","Type":"ContainerDied","Data":"5745a675f468ec367a19e9cfb50b19d9a797c45a7dfe6e718b4e9f4d3c4ab991"}
Jul 29 19:26:53.868261 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.867976    2566 scope.go:117] "RemoveContainer" containerID="147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545"
Jul 29 19:26:53.876712 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.876694    2566 scope.go:117] "RemoveContainer" containerID="147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545"
Jul 29 19:26:53.876956 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:26:53.876936    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545\": container with ID starting with 147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545 not found: ID does not exist" containerID="147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545"
Jul 29 19:26:53.877000 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.876965    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545"} err="failed to get container status \"147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545\": rpc error: code = NotFound desc = could not find container \"147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545\": container with ID starting with 147b70632e80ec92e389f2ea1494c48c987629f99267590acb0789175ac57545 not found: ID does not exist"
Jul 29 19:26:53.892636 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.892604    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf7d47f5c-8mlxw"]
Jul 29 19:26:53.895271 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:53.895251    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cf7d47f5c-8mlxw"]
Jul 29 19:26:55.102189 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:26:55.102156    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9eb4038b-49e5-4539-9c75-efd3e0473f6a" path="/var/lib/kubelet/pods/9eb4038b-49e5-4539-9c75-efd3e0473f6a/volumes"
Jul 29 19:27:11.242966 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.242932    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"]
Jul 29 19:27:11.243356 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.243295    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9eb4038b-49e5-4539-9c75-efd3e0473f6a" containerName="manager"
Jul 29 19:27:11.243356 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.243305    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9eb4038b-49e5-4539-9c75-efd3e0473f6a" containerName="manager"
Jul 29 19:27:11.243431 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.243369    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9eb4038b-49e5-4539-9c75-efd3e0473f6a" containerName="manager"
Jul 29 19:27:11.246264 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.246248    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"
Jul 29 19:27:11.254424 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.254402    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 29 19:27:11.258590 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.258573    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-d7wjg\""
Jul 29 19:27:11.281044 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.281016    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"]
Jul 29 19:27:11.347797 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.347770    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8x9v\" (UniqueName: \"kubernetes.io/projected/cfd9d45c-eab6-4b9f-8adc-8114a772f3ad-kube-api-access-h8x9v\") pod \"payload-pre-processing-687dbb6cf6-zx6xj\" (UID: \"cfd9d45c-eab6-4b9f-8adc-8114a772f3ad\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"
Jul 29 19:27:11.347936 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.347837    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfd9d45c-eab6-4b9f-8adc-8114a772f3ad-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-zx6xj\" (UID: \"cfd9d45c-eab6-4b9f-8adc-8114a772f3ad\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"
Jul 29 19:27:11.448838 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.448796    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8x9v\" (UniqueName: \"kubernetes.io/projected/cfd9d45c-eab6-4b9f-8adc-8114a772f3ad-kube-api-access-h8x9v\") pod \"payload-pre-processing-687dbb6cf6-zx6xj\" (UID: \"cfd9d45c-eab6-4b9f-8adc-8114a772f3ad\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"
Jul 29 19:27:11.449016 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.448922    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfd9d45c-eab6-4b9f-8adc-8114a772f3ad-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-zx6xj\" (UID: \"cfd9d45c-eab6-4b9f-8adc-8114a772f3ad\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"
Jul 29 19:27:11.449843 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.449818    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfd9d45c-eab6-4b9f-8adc-8114a772f3ad-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-zx6xj\" (UID: \"cfd9d45c-eab6-4b9f-8adc-8114a772f3ad\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"
Jul 29 19:27:11.459973 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.459944    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8x9v\" (UniqueName: \"kubernetes.io/projected/cfd9d45c-eab6-4b9f-8adc-8114a772f3ad-kube-api-access-h8x9v\") pod \"payload-pre-processing-687dbb6cf6-zx6xj\" (UID: \"cfd9d45c-eab6-4b9f-8adc-8114a772f3ad\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"
Jul 29 19:27:11.556341 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.556260    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"
Jul 29 19:27:11.687828 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.687798    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"]
Jul 29 19:27:11.690841 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:27:11.690812    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd9d45c_eab6_4b9f_8adc_8114a772f3ad.slice/crio-8ae42e23f6b162105a2d88aa5dac9c4b30a4ecb6851a4e78f2c4d2cc09e48b7c WatchSource:0}: Error finding container 8ae42e23f6b162105a2d88aa5dac9c4b30a4ecb6851a4e78f2c4d2cc09e48b7c: Status 404 returned error can't find the container with id 8ae42e23f6b162105a2d88aa5dac9c4b30a4ecb6851a4e78f2c4d2cc09e48b7c
Jul 29 19:27:11.954130 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:11.954095    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj" event={"ID":"cfd9d45c-eab6-4b9f-8adc-8114a772f3ad","Type":"ContainerStarted","Data":"8ae42e23f6b162105a2d88aa5dac9c4b30a4ecb6851a4e78f2c4d2cc09e48b7c"}
Jul 29 19:27:13.965528 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:13.965490    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj" event={"ID":"cfd9d45c-eab6-4b9f-8adc-8114a772f3ad","Type":"ContainerStarted","Data":"91243bae41106fd7aee4b690c1169909f9f0e088a4e02978b9aa1270e4452b97"}
Jul 29 19:27:13.965927 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:13.965616    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"
Jul 29 19:27:13.983186 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:13.983137    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj" podStartSLOduration=1.637862996 podStartE2EDuration="2.983121702s" totalImagesPullingTime="1.345258706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:27:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:27:13.981579848 +0000 UTC m=+599.477341195" watchObservedRunningTime="2026-07-29 19:27:13.983121702 +0000 UTC m=+599.478883043"
Jul 29 19:27:24.972819 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:24.972785    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-zx6xj"
Jul 29 19:27:35.054869 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.054834    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"]
Jul 29 19:27:35.059149 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.059127    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.061816 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.061794    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 19:27:35.063060 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.063039    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 29 19:27:35.063164 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.063045    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 19:27:35.063164 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.063088    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-452bp\""
Jul 29 19:27:35.067986 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.067959    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"]
Jul 29 19:27:35.164085 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.164054    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65016c35-25d7-49a8-9868-8b41aaae058c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.164085 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.164088    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.164331 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.164123    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52nsr\" (UniqueName: \"kubernetes.io/projected/65016c35-25d7-49a8-9868-8b41aaae058c-kube-api-access-52nsr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.164331 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.164151    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.164331 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.164285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.164501 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.164330    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.265473 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.265421    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65016c35-25d7-49a8-9868-8b41aaae058c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.265473 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.265465    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.265660 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.265591    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52nsr\" (UniqueName: \"kubernetes.io/projected/65016c35-25d7-49a8-9868-8b41aaae058c-kube-api-access-52nsr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.265660 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.265625    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.265751 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.265674    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.265751 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.265709    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.265887 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.265794    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.266019 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.265999    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.266080 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.266026    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.267872 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.267849    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65016c35-25d7-49a8-9868-8b41aaae058c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.268128 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.267894    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65016c35-25d7-49a8-9868-8b41aaae058c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.273708 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.273688    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52nsr\" (UniqueName: \"kubernetes.io/projected/65016c35-25d7-49a8-9868-8b41aaae058c-kube-api-access-52nsr\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zcsnb\" (UID: \"65016c35-25d7-49a8-9868-8b41aaae058c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.372145 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.372052    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:35.510530 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.510508    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"]
Jul 29 19:27:35.513275 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:27:35.513247    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65016c35_25d7_49a8_9868_8b41aaae058c.slice/crio-7051aa1dae91c3f7b4ee8a15084cfb17347f1be762a65b7fc15dcc5c3012c6e2 WatchSource:0}: Error finding container 7051aa1dae91c3f7b4ee8a15084cfb17347f1be762a65b7fc15dcc5c3012c6e2: Status 404 returned error can't find the container with id 7051aa1dae91c3f7b4ee8a15084cfb17347f1be762a65b7fc15dcc5c3012c6e2
Jul 29 19:27:35.514651 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:35.514634    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 19:27:36.072910 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:36.072871    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb" event={"ID":"65016c35-25d7-49a8-9868-8b41aaae058c","Type":"ContainerStarted","Data":"7051aa1dae91c3f7b4ee8a15084cfb17347f1be762a65b7fc15dcc5c3012c6e2"}
Jul 29 19:27:37.079382 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:37.079344    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb" event={"ID":"65016c35-25d7-49a8-9868-8b41aaae058c","Type":"ContainerStarted","Data":"171de0da91258a6a2551eb29194ea51d9282be735be728ee5453564db0f779f8"}
Jul 29 19:27:37.105777 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:37.104972    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb" podStartSLOduration=1.015306151 podStartE2EDuration="2.104945765s" totalImagesPullingTime="1.089639614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:27:37.102913228 +0000 UTC m=+622.598674570" watchObservedRunningTime="2026-07-29 19:27:37.104945765 +0000 UTC m=+622.600707108"
Jul 29 19:27:45.373029 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:45.372996    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:45.373397 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:45.373100    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:45.385896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:45.385875    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:27:46.130932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:27:46.130902    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zcsnb"
Jul 29 19:28:26.765594 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:26.765557    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"]
Jul 29 19:28:26.769624 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:26.769601    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:26.772209 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:26.772190    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 29 19:28:26.777653 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:26.777630    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"]
Jul 29 19:28:26.927161 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:26.927131    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz2rp\" (UniqueName: \"kubernetes.io/projected/1a82a835-a0d8-45c7-834c-350e95babe7f-kube-api-access-kz2rp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:26.927317 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:26.927233    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:26.927317 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:26.927258    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:26.927317 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:26.927282    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:26.927423 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:26.927356    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a82a835-a0d8-45c7-834c-350e95babe7f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:26.927423 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:26.927389    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.028675 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.028596    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.028675 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.028639    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.028876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.028755    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.028876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.028803    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a82a835-a0d8-45c7-834c-350e95babe7f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.028876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.028829    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.029038 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.028890    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz2rp\" (UniqueName: \"kubernetes.io/projected/1a82a835-a0d8-45c7-834c-350e95babe7f-kube-api-access-kz2rp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.029038 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.028947    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.029207 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.029095    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.029207 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.029184    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.030886 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.030865    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a82a835-a0d8-45c7-834c-350e95babe7f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.031178 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.031158    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a82a835-a0d8-45c7-834c-350e95babe7f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.038002 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.037977    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz2rp\" (UniqueName: \"kubernetes.io/projected/1a82a835-a0d8-45c7-834c-350e95babe7f-kube-api-access-kz2rp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk\" (UID: \"1a82a835-a0d8-45c7-834c-350e95babe7f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.079302 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.079275    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:27.215183 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.215157    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"]
Jul 29 19:28:27.217293 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:28:27.217263    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a82a835_a0d8_45c7_834c_350e95babe7f.slice/crio-fa6fd356d1a765e0478b82b3dbdabef3bf4f2580c6e35f7cb91dfe2f8c0a2993 WatchSource:0}: Error finding container fa6fd356d1a765e0478b82b3dbdabef3bf4f2580c6e35f7cb91dfe2f8c0a2993: Status 404 returned error can't find the container with id fa6fd356d1a765e0478b82b3dbdabef3bf4f2580c6e35f7cb91dfe2f8c0a2993
Jul 29 19:28:27.317155 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:27.317081    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk" event={"ID":"1a82a835-a0d8-45c7-834c-350e95babe7f","Type":"ContainerStarted","Data":"fa6fd356d1a765e0478b82b3dbdabef3bf4f2580c6e35f7cb91dfe2f8c0a2993"}
Jul 29 19:28:28.323488 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:28.323380    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk" event={"ID":"1a82a835-a0d8-45c7-834c-350e95babe7f","Type":"ContainerStarted","Data":"e8201217e3df7526cdade379a0570ab6c9b749e466421bab4582c8ae93090825"}
Jul 29 19:28:28.345557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:28.345510    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk" podStartSLOduration=1.5759895369999999 podStartE2EDuration="2.345490833s" totalImagesPullingTime="769.501296ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:28:28.343852839 +0000 UTC m=+673.839614197" watchObservedRunningTime="2026-07-29 19:28:28.345490833 +0000 UTC m=+673.841252178"
Jul 29 19:28:37.080180 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:37.080133    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:37.080180 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:37.080194    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:37.092198 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:37.092168    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:28:37.378885 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:28:37.378812    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk"
Jul 29 19:30:01.102860 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.102828    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b78748b79-vk5tq"]
Jul 29 19:30:01.105939 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.105903    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.114316 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.114284    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b78748b79-vk5tq"]
Jul 29 19:30:01.157643 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.157607    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-console-config\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.157793 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.157659    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-oauth-serving-cert\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.157793 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.157724    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-trusted-ca-bundle\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.157793 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.157759    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdfrv\" (UniqueName: \"kubernetes.io/projected/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-kube-api-access-jdfrv\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.157793 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.157782    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-service-ca\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.157949 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.157863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-console-oauth-config\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.157949 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.157898    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-console-serving-cert\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.259019 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.258988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-trusted-ca-bundle\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.259166 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.259033    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdfrv\" (UniqueName: \"kubernetes.io/projected/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-kube-api-access-jdfrv\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.259166 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.259054    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-service-ca\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.259166 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.259090    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-console-oauth-config\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.259166 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.259133    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-console-serving-cert\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.259166 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.259162    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-console-config\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.259431 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.259191    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-oauth-serving-cert\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.259882 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.259848    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-service-ca\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.260003 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.259899    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-console-config\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.260003 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.259912    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-trusted-ca-bundle\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.260003 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.259923    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-oauth-serving-cert\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.261587 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.261557    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-console-serving-cert\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.261677 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.261607    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-console-oauth-config\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.271171 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.271153    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdfrv\" (UniqueName: \"kubernetes.io/projected/cd2f9792-f42f-49a9-9a9b-c33cea2911fc-kube-api-access-jdfrv\") pod \"console-b78748b79-vk5tq\" (UID: \"cd2f9792-f42f-49a9-9a9b-c33cea2911fc\") " pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.416673 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.416575    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:01.553674 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.553650    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b78748b79-vk5tq"]
Jul 29 19:30:01.555365 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:30:01.555329    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2f9792_f42f_49a9_9a9b_c33cea2911fc.slice/crio-dcb6556af5f564b034686ab1a71f48d3572a0f4708b17a779bfd6379c1bbaf39 WatchSource:0}: Error finding container dcb6556af5f564b034686ab1a71f48d3572a0f4708b17a779bfd6379c1bbaf39: Status 404 returned error can't find the container with id dcb6556af5f564b034686ab1a71f48d3572a0f4708b17a779bfd6379c1bbaf39
Jul 29 19:30:01.770619 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.770583    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b78748b79-vk5tq" event={"ID":"cd2f9792-f42f-49a9-9a9b-c33cea2911fc","Type":"ContainerStarted","Data":"5b93a63f0564386180f647a1f34e0b259eb978e6016719887456341c51d2a3a0"}
Jul 29 19:30:01.770791 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.770640    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b78748b79-vk5tq" event={"ID":"cd2f9792-f42f-49a9-9a9b-c33cea2911fc","Type":"ContainerStarted","Data":"dcb6556af5f564b034686ab1a71f48d3572a0f4708b17a779bfd6379c1bbaf39"}
Jul 29 19:30:01.791722 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:01.791677    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b78748b79-vk5tq" podStartSLOduration=0.791661938 podStartE2EDuration="791.661938ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:01.790782156 +0000 UTC m=+767.286543493" watchObservedRunningTime="2026-07-29 19:30:01.791661938 +0000 UTC m=+767.287423351"
Jul 29 19:30:11.417269 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:11.417231    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:11.417765 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:11.417298    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:11.421579 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:11.421559    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:11.822603 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:11.822575    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b78748b79-vk5tq"
Jul 29 19:30:11.877992 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:11.877959    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d96749bc4-44lbb"]
Jul 29 19:30:36.897251 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:36.897144    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d96749bc4-44lbb" podUID="db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" containerName="console" containerID="cri-o://1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d" gracePeriod=15
Jul 29 19:30:37.161803 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.161746    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:30:37.300465 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.300418    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-serving-cert\") pod \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") "
Jul 29 19:30:37.300651 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.300479    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bg259\" (UniqueName: \"kubernetes.io/projected/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-kube-api-access-bg259\") pod \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") "
Jul 29 19:30:37.300651 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.300501    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-oauth-config\") pod \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") "
Jul 29 19:30:37.300651 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.300528    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-service-ca\") pod \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") "
Jul 29 19:30:37.300651 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.300619    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-trusted-ca-bundle\") pod \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") "
Jul 29 19:30:37.300865 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.300674    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-oauth-serving-cert\") pod \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") "
Jul 29 19:30:37.300865 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.300716    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-config\") pod \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\" (UID: \"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18\") "
Jul 29 19:30:37.301203 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.301124    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-service-ca" (OuterVolumeSpecName: "service-ca") pod "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" (UID: "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:30:37.301203 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.301131    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" (UID: "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:30:37.301203 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.301140    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" (UID: "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:30:37.301407 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.301251    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-config" (OuterVolumeSpecName: "console-config") pod "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" (UID: "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:30:37.302695 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.302665    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-kube-api-access-bg259" (OuterVolumeSpecName: "kube-api-access-bg259") pod "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" (UID: "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18"). InnerVolumeSpecName "kube-api-access-bg259". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:30:37.302695 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.302686    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" (UID: "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:30:37.302821 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.302726    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" (UID: "db0270d8-f0e9-4ffc-9ce5-4995d4d49f18"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:30:37.402630 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.402594    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-oauth-serving-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:30:37.402630 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.402628    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:30:37.402630 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.402638    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-serving-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:30:37.402885 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.402646    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bg259\" (UniqueName: \"kubernetes.io/projected/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-kube-api-access-bg259\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:30:37.402885 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.402657    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-console-oauth-config\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:30:37.402885 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.402666    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-service-ca\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:30:37.402885 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.402679    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18-trusted-ca-bundle\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:30:37.947086 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.947046    2566 generic.go:356] "Generic (PLEG): container finished" podID="db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" containerID="1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d" exitCode=0
Jul 29 19:30:37.947574 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.947093    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d96749bc4-44lbb"
Jul 29 19:30:37.947574 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.947135    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d96749bc4-44lbb" event={"ID":"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18","Type":"ContainerDied","Data":"1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d"}
Jul 29 19:30:37.947574 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.947190    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d96749bc4-44lbb" event={"ID":"db0270d8-f0e9-4ffc-9ce5-4995d4d49f18","Type":"ContainerDied","Data":"56aed548c1e35c0dde4cfd11612a766f94894c0c6b4ed5f2ebf9c28e8d28ba63"}
Jul 29 19:30:37.947574 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.947212    2566 scope.go:117] "RemoveContainer" containerID="1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d"
Jul 29 19:30:37.956419 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.956398    2566 scope.go:117] "RemoveContainer" containerID="1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d"
Jul 29 19:30:37.956703 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:30:37.956685    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d\": container with ID starting with 1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d not found: ID does not exist" containerID="1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d"
Jul 29 19:30:37.956761 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.956713    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d"} err="failed to get container status \"1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d\": rpc error: code = NotFound desc = could not find container \"1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d\": container with ID starting with 1e37a291074199cda102e77e916550f7ed7a25d6b4befcd6e1e445d60ce6380d not found: ID does not exist"
Jul 29 19:30:37.970808 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.970779    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d96749bc4-44lbb"]
Jul 29 19:30:37.978841 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:37.978815    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d96749bc4-44lbb"]
Jul 29 19:30:39.102435 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:39.102400    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" path="/var/lib/kubelet/pods/db0270d8-f0e9-4ffc-9ce5-4995d4d49f18/volumes"
Jul 29 19:30:48.819999 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:48.819959    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-k6hfg"]
Jul 29 19:30:48.820636 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:48.820230    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65d7444bfc-k6hfg" podUID="ca2246db-ea6b-4d75-8667-0da5ece52c58" containerName="manager" containerID="cri-o://d76f1a648f08513df1214c0a9d4a513c7507a8ff7bf9de8d38cf8898910c0145" gracePeriod=30
Jul 29 19:30:49.001636 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:49.001605    2566 generic.go:356] "Generic (PLEG): container finished" podID="ca2246db-ea6b-4d75-8667-0da5ece52c58" containerID="d76f1a648f08513df1214c0a9d4a513c7507a8ff7bf9de8d38cf8898910c0145" exitCode=0
Jul 29 19:30:49.001763 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:49.001644    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d7444bfc-k6hfg" event={"ID":"ca2246db-ea6b-4d75-8667-0da5ece52c58","Type":"ContainerDied","Data":"d76f1a648f08513df1214c0a9d4a513c7507a8ff7bf9de8d38cf8898910c0145"}
Jul 29 19:30:49.072069 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:49.072003    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:30:49.100275 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:49.100251    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7td9\" (UniqueName: \"kubernetes.io/projected/ca2246db-ea6b-4d75-8667-0da5ece52c58-kube-api-access-l7td9\") pod \"ca2246db-ea6b-4d75-8667-0da5ece52c58\" (UID: \"ca2246db-ea6b-4d75-8667-0da5ece52c58\") "
Jul 29 19:30:49.100437 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:49.100310    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca2246db-ea6b-4d75-8667-0da5ece52c58-webhook-cert\") pod \"ca2246db-ea6b-4d75-8667-0da5ece52c58\" (UID: \"ca2246db-ea6b-4d75-8667-0da5ece52c58\") "
Jul 29 19:30:49.102387 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:49.102354    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca2246db-ea6b-4d75-8667-0da5ece52c58-kube-api-access-l7td9" (OuterVolumeSpecName: "kube-api-access-l7td9") pod "ca2246db-ea6b-4d75-8667-0da5ece52c58" (UID: "ca2246db-ea6b-4d75-8667-0da5ece52c58"). InnerVolumeSpecName "kube-api-access-l7td9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:30:49.102880 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:49.102853    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca2246db-ea6b-4d75-8667-0da5ece52c58-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ca2246db-ea6b-4d75-8667-0da5ece52c58" (UID: "ca2246db-ea6b-4d75-8667-0da5ece52c58"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:30:49.201826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:49.201799    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7td9\" (UniqueName: \"kubernetes.io/projected/ca2246db-ea6b-4d75-8667-0da5ece52c58-kube-api-access-l7td9\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:30:49.201826 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:49.201825    2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca2246db-ea6b-4d75-8667-0da5ece52c58-webhook-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:30:50.008139 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:50.008098    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d7444bfc-k6hfg" event={"ID":"ca2246db-ea6b-4d75-8667-0da5ece52c58","Type":"ContainerDied","Data":"04f991760c8a96b66fb1687c476720b026b2c3651b889accd026a970dd229da6"}
Jul 29 19:30:50.008139 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:50.008121    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d7444bfc-k6hfg"
Jul 29 19:30:50.008645 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:50.008169    2566 scope.go:117] "RemoveContainer" containerID="d76f1a648f08513df1214c0a9d4a513c7507a8ff7bf9de8d38cf8898910c0145"
Jul 29 19:30:50.028188 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:50.028155    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-k6hfg"]
Jul 29 19:30:50.034512 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:50.033143    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-k6hfg"]
Jul 29 19:30:51.102102 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:30:51.102070    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca2246db-ea6b-4d75-8667-0da5ece52c58" path="/var/lib/kubelet/pods/ca2246db-ea6b-4d75-8667-0da5ece52c58/volumes"
Jul 29 19:31:50.810385 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.810350    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65d7444bfc-khc59"]
Jul 29 19:31:50.810957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.810803    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" containerName="console"
Jul 29 19:31:50.810957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.810821    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" containerName="console"
Jul 29 19:31:50.810957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.810853    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca2246db-ea6b-4d75-8667-0da5ece52c58" containerName="manager"
Jul 29 19:31:50.810957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.810863    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca2246db-ea6b-4d75-8667-0da5ece52c58" containerName="manager"
Jul 29 19:31:50.810957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.810947    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="db0270d8-f0e9-4ffc-9ce5-4995d4d49f18" containerName="console"
Jul 29 19:31:50.810957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.810956    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca2246db-ea6b-4d75-8667-0da5ece52c58" containerName="manager"
Jul 29 19:31:50.814057 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.814036    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:31:50.817916 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.817895    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d9slh\""
Jul 29 19:31:50.818048 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.817897    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 19:31:50.818048 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.817926    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 19:31:50.828200 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.828180    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-khc59"]
Jul 29 19:31:50.890331 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.890303    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n775t\" (UniqueName: \"kubernetes.io/projected/db732ac2-60f0-4485-bb19-5a349bd27f55-kube-api-access-n775t\") pod \"maas-controller-65d7444bfc-khc59\" (UID: \"db732ac2-60f0-4485-bb19-5a349bd27f55\") " pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:31:50.890504 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.890406    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db732ac2-60f0-4485-bb19-5a349bd27f55-webhook-cert\") pod \"maas-controller-65d7444bfc-khc59\" (UID: \"db732ac2-60f0-4485-bb19-5a349bd27f55\") " pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:31:50.991122 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.991092    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db732ac2-60f0-4485-bb19-5a349bd27f55-webhook-cert\") pod \"maas-controller-65d7444bfc-khc59\" (UID: \"db732ac2-60f0-4485-bb19-5a349bd27f55\") " pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:31:50.991271 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.991153    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n775t\" (UniqueName: \"kubernetes.io/projected/db732ac2-60f0-4485-bb19-5a349bd27f55-kube-api-access-n775t\") pod \"maas-controller-65d7444bfc-khc59\" (UID: \"db732ac2-60f0-4485-bb19-5a349bd27f55\") " pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:31:50.993471 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:50.993439    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db732ac2-60f0-4485-bb19-5a349bd27f55-webhook-cert\") pod \"maas-controller-65d7444bfc-khc59\" (UID: \"db732ac2-60f0-4485-bb19-5a349bd27f55\") " pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:31:51.000490 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:51.000420    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n775t\" (UniqueName: \"kubernetes.io/projected/db732ac2-60f0-4485-bb19-5a349bd27f55-kube-api-access-n775t\") pod \"maas-controller-65d7444bfc-khc59\" (UID: \"db732ac2-60f0-4485-bb19-5a349bd27f55\") " pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:31:51.123696 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:51.123619    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:31:51.255585 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:51.255561    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-khc59"]
Jul 29 19:31:51.257122 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:31:51.257092    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb732ac2_60f0_4485_bb19_5a349bd27f55.slice/crio-b676beb6650648600f98ea87a72573deab785a48f551b0deb0b86fc1dff06a4d WatchSource:0}: Error finding container b676beb6650648600f98ea87a72573deab785a48f551b0deb0b86fc1dff06a4d: Status 404 returned error can't find the container with id b676beb6650648600f98ea87a72573deab785a48f551b0deb0b86fc1dff06a4d
Jul 29 19:31:51.304175 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:51.304149    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d7444bfc-khc59" event={"ID":"db732ac2-60f0-4485-bb19-5a349bd27f55","Type":"ContainerStarted","Data":"b676beb6650648600f98ea87a72573deab785a48f551b0deb0b86fc1dff06a4d"}
Jul 29 19:31:52.310627 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:52.310590    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d7444bfc-khc59" event={"ID":"db732ac2-60f0-4485-bb19-5a349bd27f55","Type":"ContainerStarted","Data":"d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae"}
Jul 29 19:31:52.311017 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:52.310656    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:31:52.345682 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:31:52.345627    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65d7444bfc-khc59" podStartSLOduration=1.916099947 podStartE2EDuration="2.345611286s" totalImagesPullingTime="429.511339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:31:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:31:52.344437363 +0000 UTC m=+877.840198732" watchObservedRunningTime="2026-07-29 19:31:52.345611286 +0000 UTC m=+877.841372791"
Jul 29 19:32:03.317502 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:03.317442    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:32:07.966187 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:07.966152    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-khc59"]
Jul 29 19:32:07.966642 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:07.966344    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65d7444bfc-khc59" podUID="db732ac2-60f0-4485-bb19-5a349bd27f55" containerName="manager" containerID="cri-o://d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae" gracePeriod=30
Jul 29 19:32:08.217382 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.217324    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:32:08.344837 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.344807    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n775t\" (UniqueName: \"kubernetes.io/projected/db732ac2-60f0-4485-bb19-5a349bd27f55-kube-api-access-n775t\") pod \"db732ac2-60f0-4485-bb19-5a349bd27f55\" (UID: \"db732ac2-60f0-4485-bb19-5a349bd27f55\") "
Jul 29 19:32:08.345022 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.344854    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db732ac2-60f0-4485-bb19-5a349bd27f55-webhook-cert\") pod \"db732ac2-60f0-4485-bb19-5a349bd27f55\" (UID: \"db732ac2-60f0-4485-bb19-5a349bd27f55\") "
Jul 29 19:32:08.346795 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.346762    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db732ac2-60f0-4485-bb19-5a349bd27f55-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "db732ac2-60f0-4485-bb19-5a349bd27f55" (UID: "db732ac2-60f0-4485-bb19-5a349bd27f55"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:32:08.346916 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.346866    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db732ac2-60f0-4485-bb19-5a349bd27f55-kube-api-access-n775t" (OuterVolumeSpecName: "kube-api-access-n775t") pod "db732ac2-60f0-4485-bb19-5a349bd27f55" (UID: "db732ac2-60f0-4485-bb19-5a349bd27f55"). InnerVolumeSpecName "kube-api-access-n775t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:32:08.388377 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.388346    2566 generic.go:356] "Generic (PLEG): container finished" podID="db732ac2-60f0-4485-bb19-5a349bd27f55" containerID="d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae" exitCode=0
Jul 29 19:32:08.388538 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.388387    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d7444bfc-khc59"
Jul 29 19:32:08.388538 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.388438    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d7444bfc-khc59" event={"ID":"db732ac2-60f0-4485-bb19-5a349bd27f55","Type":"ContainerDied","Data":"d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae"}
Jul 29 19:32:08.388538 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.388528    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d7444bfc-khc59" event={"ID":"db732ac2-60f0-4485-bb19-5a349bd27f55","Type":"ContainerDied","Data":"b676beb6650648600f98ea87a72573deab785a48f551b0deb0b86fc1dff06a4d"}
Jul 29 19:32:08.388716 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.388551    2566 scope.go:117] "RemoveContainer" containerID="d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae"
Jul 29 19:32:08.396964 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.396945    2566 scope.go:117] "RemoveContainer" containerID="d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae"
Jul 29 19:32:08.397222 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:32:08.397202    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae\": container with ID starting with d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae not found: ID does not exist" containerID="d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae"
Jul 29 19:32:08.397281 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.397229    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae"} err="failed to get container status \"d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae\": rpc error: code = NotFound desc = could not find container \"d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae\": container with ID starting with d5362b2be94df0e696e7329e8f78c995ae42cb7d3c7537a30d7569b0e03887ae not found: ID does not exist"
Jul 29 19:32:08.411163 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.411143    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-khc59"]
Jul 29 19:32:08.415020 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.414994    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-khc59"]
Jul 29 19:32:08.445961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.445938    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n775t\" (UniqueName: \"kubernetes.io/projected/db732ac2-60f0-4485-bb19-5a349bd27f55-kube-api-access-n775t\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:32:08.445961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:08.445958    2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db732ac2-60f0-4485-bb19-5a349bd27f55-webhook-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:32:09.101789 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.101756    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db732ac2-60f0-4485-bb19-5a349bd27f55" path="/var/lib/kubelet/pods/db732ac2-60f0-4485-bb19-5a349bd27f55/volumes"
Jul 29 19:32:09.213857 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.213823    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65d7444bfc-2rmm5"]
Jul 29 19:32:09.214274 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.214257    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db732ac2-60f0-4485-bb19-5a349bd27f55" containerName="manager"
Jul 29 19:32:09.214363 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.214277    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="db732ac2-60f0-4485-bb19-5a349bd27f55" containerName="manager"
Jul 29 19:32:09.214415 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.214395    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="db732ac2-60f0-4485-bb19-5a349bd27f55" containerName="manager"
Jul 29 19:32:09.218750 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.218728    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d7444bfc-2rmm5"
Jul 29 19:32:09.222795 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.222772    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 19:32:09.222921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.222780    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 19:32:09.222921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.222810    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d9slh\""
Jul 29 19:32:09.229133 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.229112    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-2rmm5"]
Jul 29 19:32:09.355482 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.355375    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grxxp\" (UniqueName: \"kubernetes.io/projected/41a9ba4e-6393-425e-a00b-da3d4494f889-kube-api-access-grxxp\") pod \"maas-controller-65d7444bfc-2rmm5\" (UID: \"41a9ba4e-6393-425e-a00b-da3d4494f889\") " pod="opendatahub/maas-controller-65d7444bfc-2rmm5"
Jul 29 19:32:09.355482 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.355417    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41a9ba4e-6393-425e-a00b-da3d4494f889-webhook-cert\") pod \"maas-controller-65d7444bfc-2rmm5\" (UID: \"41a9ba4e-6393-425e-a00b-da3d4494f889\") " pod="opendatahub/maas-controller-65d7444bfc-2rmm5"
Jul 29 19:32:09.456613 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.456577    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grxxp\" (UniqueName: \"kubernetes.io/projected/41a9ba4e-6393-425e-a00b-da3d4494f889-kube-api-access-grxxp\") pod \"maas-controller-65d7444bfc-2rmm5\" (UID: \"41a9ba4e-6393-425e-a00b-da3d4494f889\") " pod="opendatahub/maas-controller-65d7444bfc-2rmm5"
Jul 29 19:32:09.456801 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.456630    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41a9ba4e-6393-425e-a00b-da3d4494f889-webhook-cert\") pod \"maas-controller-65d7444bfc-2rmm5\" (UID: \"41a9ba4e-6393-425e-a00b-da3d4494f889\") " pod="opendatahub/maas-controller-65d7444bfc-2rmm5"
Jul 29 19:32:09.459032 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.459005    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/41a9ba4e-6393-425e-a00b-da3d4494f889-webhook-cert\") pod \"maas-controller-65d7444bfc-2rmm5\" (UID: \"41a9ba4e-6393-425e-a00b-da3d4494f889\") " pod="opendatahub/maas-controller-65d7444bfc-2rmm5"
Jul 29 19:32:09.466251 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.466230    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grxxp\" (UniqueName: \"kubernetes.io/projected/41a9ba4e-6393-425e-a00b-da3d4494f889-kube-api-access-grxxp\") pod \"maas-controller-65d7444bfc-2rmm5\" (UID: \"41a9ba4e-6393-425e-a00b-da3d4494f889\") " pod="opendatahub/maas-controller-65d7444bfc-2rmm5"
Jul 29 19:32:09.529505 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.529471    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d7444bfc-2rmm5"
Jul 29 19:32:09.661277 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:09.661254    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d7444bfc-2rmm5"]
Jul 29 19:32:09.663262 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:32:09.663237    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a9ba4e_6393_425e_a00b_da3d4494f889.slice/crio-e518c44e247752d66d536d481de5f38bb5634fa0362bb037902c3e487fea8206 WatchSource:0}: Error finding container e518c44e247752d66d536d481de5f38bb5634fa0362bb037902c3e487fea8206: Status 404 returned error can't find the container with id e518c44e247752d66d536d481de5f38bb5634fa0362bb037902c3e487fea8206
Jul 29 19:32:10.400743 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:10.400660    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d7444bfc-2rmm5" event={"ID":"41a9ba4e-6393-425e-a00b-da3d4494f889","Type":"ContainerStarted","Data":"4a128e3d8ac774595b34ba2b35dd6f16df5c5027a3e28d24d4ef552311dc533b"}
Jul 29 19:32:10.400743 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:10.400723    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d7444bfc-2rmm5" event={"ID":"41a9ba4e-6393-425e-a00b-da3d4494f889","Type":"ContainerStarted","Data":"e518c44e247752d66d536d481de5f38bb5634fa0362bb037902c3e487fea8206"}
Jul 29 19:32:10.401109 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:10.400758    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65d7444bfc-2rmm5"
Jul 29 19:32:10.421318 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:10.421271    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65d7444bfc-2rmm5" podStartSLOduration=1.09572293 podStartE2EDuration="1.421256383s" totalImagesPullingTime="325.533453ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:32:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:32:10.418479631 +0000 UTC m=+895.914241000" watchObservedRunningTime="2026-07-29 19:32:10.421256383 +0000 UTC m=+895.917017724"
Jul 29 19:32:21.407350 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:32:21.407319    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65d7444bfc-2rmm5"
Jul 29 19:37:06.493370 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:06.493331    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"]
Jul 29 19:37:06.493979 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:06.493671    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb" podUID="18f8dfea-9c84-44d0-801b-2ee7763b8af4" containerName="manager" containerID="cri-o://fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268" gracePeriod=10
Jul 29 19:37:09.352149 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.352123    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:37:09.474988 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.474956    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd5c4\" (UniqueName: \"kubernetes.io/projected/18f8dfea-9c84-44d0-801b-2ee7763b8af4-kube-api-access-jd5c4\") pod \"18f8dfea-9c84-44d0-801b-2ee7763b8af4\" (UID: \"18f8dfea-9c84-44d0-801b-2ee7763b8af4\") "
Jul 29 19:37:09.475175 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.475052    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18f8dfea-9c84-44d0-801b-2ee7763b8af4-extensions-socket-volume\") pod \"18f8dfea-9c84-44d0-801b-2ee7763b8af4\" (UID: \"18f8dfea-9c84-44d0-801b-2ee7763b8af4\") "
Jul 29 19:37:09.475396 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.475371    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18f8dfea-9c84-44d0-801b-2ee7763b8af4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "18f8dfea-9c84-44d0-801b-2ee7763b8af4" (UID: "18f8dfea-9c84-44d0-801b-2ee7763b8af4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:37:09.477087 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.477068    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f8dfea-9c84-44d0-801b-2ee7763b8af4-kube-api-access-jd5c4" (OuterVolumeSpecName: "kube-api-access-jd5c4") pod "18f8dfea-9c84-44d0-801b-2ee7763b8af4" (UID: "18f8dfea-9c84-44d0-801b-2ee7763b8af4"). InnerVolumeSpecName "kube-api-access-jd5c4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:37:09.576432 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.576397    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd5c4\" (UniqueName: \"kubernetes.io/projected/18f8dfea-9c84-44d0-801b-2ee7763b8af4-kube-api-access-jd5c4\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:37:09.576432 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.576423    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18f8dfea-9c84-44d0-801b-2ee7763b8af4-extensions-socket-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:37:09.822054 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.821966    2566 generic.go:356] "Generic (PLEG): container finished" podID="18f8dfea-9c84-44d0-801b-2ee7763b8af4" containerID="fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268" exitCode=0
Jul 29 19:37:09.822054 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.822010    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"
Jul 29 19:37:09.822221 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.822056    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb" event={"ID":"18f8dfea-9c84-44d0-801b-2ee7763b8af4","Type":"ContainerDied","Data":"fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268"}
Jul 29 19:37:09.822221 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.822111    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb" event={"ID":"18f8dfea-9c84-44d0-801b-2ee7763b8af4","Type":"ContainerDied","Data":"2069ace9d0b57aa4fa3f279de28a51c127cbc5524a94bd68e797126cff05bd7f"}
Jul 29 19:37:09.822221 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.822140    2566 scope.go:117] "RemoveContainer" containerID="fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268"
Jul 29 19:37:09.831757 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.831740    2566 scope.go:117] "RemoveContainer" containerID="fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268"
Jul 29 19:37:09.831991 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:37:09.831972    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268\": container with ID starting with fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268 not found: ID does not exist" containerID="fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268"
Jul 29 19:37:09.832062 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.832005    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268"} err="failed to get container status \"fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268\": rpc error: code = NotFound desc = could not find container \"fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268\": container with ID starting with fc67de79cdbb2fd4ae49e9f38fd0e4ecb8c9d2413857df8f88f74d5a76a52268 not found: ID does not exist"
Jul 29 19:37:09.845906 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.845882    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"]
Jul 29 19:37:09.849873 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:09.849849    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q24zb"]
Jul 29 19:37:11.101243 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:37:11.101209    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18f8dfea-9c84-44d0-801b-2ee7763b8af4" path="/var/lib/kubelet/pods/18f8dfea-9c84-44d0-801b-2ee7763b8af4/volumes"
Jul 29 19:38:07.582516 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.582473    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"]
Jul 29 19:38:07.584033 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.583031    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18f8dfea-9c84-44d0-801b-2ee7763b8af4" containerName="manager"
Jul 29 19:38:07.584033 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.583047    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="18f8dfea-9c84-44d0-801b-2ee7763b8af4" containerName="manager"
Jul 29 19:38:07.584033 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.583104    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="18f8dfea-9c84-44d0-801b-2ee7763b8af4" containerName="manager"
Jul 29 19:38:07.585040 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.585020    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"
Jul 29 19:38:07.587716 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.587698    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j5g8m\""
Jul 29 19:38:07.597189 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.597164    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"]
Jul 29 19:38:07.719669 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.719634    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d985cbc-050d-4348-97c8-b90aed917660-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2f8n5\" (UID: \"6d985cbc-050d-4348-97c8-b90aed917660\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"
Jul 29 19:38:07.719669 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.719673    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh89x\" (UniqueName: \"kubernetes.io/projected/6d985cbc-050d-4348-97c8-b90aed917660-kube-api-access-fh89x\") pod \"kuadrant-operator-controller-manager-86878c769f-2f8n5\" (UID: \"6d985cbc-050d-4348-97c8-b90aed917660\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"
Jul 29 19:38:07.821074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.821036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d985cbc-050d-4348-97c8-b90aed917660-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2f8n5\" (UID: \"6d985cbc-050d-4348-97c8-b90aed917660\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"
Jul 29 19:38:07.821074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.821076    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh89x\" (UniqueName: \"kubernetes.io/projected/6d985cbc-050d-4348-97c8-b90aed917660-kube-api-access-fh89x\") pod \"kuadrant-operator-controller-manager-86878c769f-2f8n5\" (UID: \"6d985cbc-050d-4348-97c8-b90aed917660\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"
Jul 29 19:38:07.821490 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.821472    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6d985cbc-050d-4348-97c8-b90aed917660-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2f8n5\" (UID: \"6d985cbc-050d-4348-97c8-b90aed917660\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"
Jul 29 19:38:07.831468 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.831428    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh89x\" (UniqueName: \"kubernetes.io/projected/6d985cbc-050d-4348-97c8-b90aed917660-kube-api-access-fh89x\") pod \"kuadrant-operator-controller-manager-86878c769f-2f8n5\" (UID: \"6d985cbc-050d-4348-97c8-b90aed917660\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"
Jul 29 19:38:07.895519 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:07.895425    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"
Jul 29 19:38:08.036652 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:08.036624    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"]
Jul 29 19:38:08.039537 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:38:08.039513    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d985cbc_050d_4348_97c8_b90aed917660.slice/crio-1288bbf39bfe82bb3002f40cc4eafadabf8a3596045346a2fab8788f68e00211 WatchSource:0}: Error finding container 1288bbf39bfe82bb3002f40cc4eafadabf8a3596045346a2fab8788f68e00211: Status 404 returned error can't find the container with id 1288bbf39bfe82bb3002f40cc4eafadabf8a3596045346a2fab8788f68e00211
Jul 29 19:38:08.095784 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:08.095754    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5" event={"ID":"6d985cbc-050d-4348-97c8-b90aed917660","Type":"ContainerStarted","Data":"1288bbf39bfe82bb3002f40cc4eafadabf8a3596045346a2fab8788f68e00211"}
Jul 29 19:38:09.102566 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:09.102536    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"
Jul 29 19:38:09.102945 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:09.102580    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5" event={"ID":"6d985cbc-050d-4348-97c8-b90aed917660","Type":"ContainerStarted","Data":"5a188db16be90e9b15e31202f030f9630b2a73b79e5196b201ef98f4ac566282"}
Jul 29 19:38:09.125678 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:09.125632    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5" podStartSLOduration=2.125615418 podStartE2EDuration="2.125615418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:38:09.124388671 +0000 UTC m=+1254.620150044" watchObservedRunningTime="2026-07-29 19:38:09.125615418 +0000 UTC m=+1254.621376761"
Jul 29 19:38:20.109413 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:38:20.109377    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2f8n5"
Jul 29 19:41:02.374761 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.374724    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"]
Jul 29 19:41:02.378169 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.378148    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.380928 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.380909    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-569c3022-kserve-self-signed-certs\""
Jul 29 19:41:02.400914 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.400887    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"]
Jul 29 19:41:02.430649 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.430618    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-model-cache\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.430812 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.430713    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-home\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.430812 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.430770    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tls-certs\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.430812 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.430809    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-dshm\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.430933 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.430870    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tmp-dir\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.430933 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.430911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4rxb\" (UniqueName: \"kubernetes.io/projected/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-kube-api-access-d4rxb\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.531412 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.531376    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-dshm\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.531620 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.531436    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tmp-dir\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.531620 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.531496    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4rxb\" (UniqueName: \"kubernetes.io/projected/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-kube-api-access-d4rxb\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.531620 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.531581    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-model-cache\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.531778 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.531738    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-home\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.531842 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.531823    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tls-certs\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.531936 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.531906    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tmp-dir\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.531936 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.531916    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-model-cache\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.532151 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.532131    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-home\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.533780 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.533753    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-dshm\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.534105 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.534086    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tls-certs\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.540427 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.540405    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4rxb\" (UniqueName: \"kubernetes.io/projected/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-kube-api-access-d4rxb\") pod \"e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") " pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.688470 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.688419    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:02.825073 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.825027    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"]
Jul 29 19:41:02.827719 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:41:02.827688    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31c0d1db_215b_4ead_bda6_4ac3a766c8c1.slice/crio-da2867d7c6e843bd605c813ab53eea042e38e62a5b44863c96af42cf3a6804a5 WatchSource:0}: Error finding container da2867d7c6e843bd605c813ab53eea042e38e62a5b44863c96af42cf3a6804a5: Status 404 returned error can't find the container with id da2867d7c6e843bd605c813ab53eea042e38e62a5b44863c96af42cf3a6804a5
Jul 29 19:41:02.928025 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.927993    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q" event={"ID":"31c0d1db-215b-4ead-bda6-4ac3a766c8c1","Type":"ContainerStarted","Data":"b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233"}
Jul 29 19:41:02.928173 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.928035    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q" event={"ID":"31c0d1db-215b-4ead-bda6-4ac3a766c8c1","Type":"ContainerStarted","Data":"da2867d7c6e843bd605c813ab53eea042e38e62a5b44863c96af42cf3a6804a5"}
Jul 29 19:41:02.947958 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:02.947857    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q" podStartSLOduration=0.947840895 podStartE2EDuration="947.840895ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:41:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:41:02.945304395 +0000 UTC m=+1428.441065759" watchObservedRunningTime="2026-07-29 19:41:02.947840895 +0000 UTC m=+1428.443602237"
Jul 29 19:41:12.688584 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:12.688547    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:12.689139 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:12.688633    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:12.701139 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:12.701106    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:12.987674 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:12.987597    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:31.217602 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:31.217571    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"]
Jul 29 19:41:46.236550 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.236473    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q" podUID="31c0d1db-215b-4ead-bda6-4ac3a766c8c1" containerName="main" containerID="cri-o://b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233" gracePeriod=45
Jul 29 19:41:46.490993 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.490926    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:46.557404 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.557376    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4rxb\" (UniqueName: \"kubernetes.io/projected/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-kube-api-access-d4rxb\") pod \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") "
Jul 29 19:41:46.557573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.557430    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-dshm\") pod \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") "
Jul 29 19:41:46.557573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.557482    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tmp-dir\") pod \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") "
Jul 29 19:41:46.557573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.557529    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-model-cache\") pod \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") "
Jul 29 19:41:46.557573 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.557565    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tls-certs\") pod \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") "
Jul 29 19:41:46.557812 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.557612    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-home\") pod \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\" (UID: \"31c0d1db-215b-4ead-bda6-4ac3a766c8c1\") "
Jul 29 19:41:46.557812 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.557789    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-model-cache" (OuterVolumeSpecName: "model-cache") pod "31c0d1db-215b-4ead-bda6-4ac3a766c8c1" (UID: "31c0d1db-215b-4ead-bda6-4ac3a766c8c1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:41:46.557914 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.557799    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "31c0d1db-215b-4ead-bda6-4ac3a766c8c1" (UID: "31c0d1db-215b-4ead-bda6-4ac3a766c8c1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:41:46.557914 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.557883    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tmp-dir\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:41:46.557914 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.557903    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-model-cache\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:41:46.558028 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.557963    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-home" (OuterVolumeSpecName: "home") pod "31c0d1db-215b-4ead-bda6-4ac3a766c8c1" (UID: "31c0d1db-215b-4ead-bda6-4ac3a766c8c1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:41:46.559685 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.559653    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-dshm" (OuterVolumeSpecName: "dshm") pod "31c0d1db-215b-4ead-bda6-4ac3a766c8c1" (UID: "31c0d1db-215b-4ead-bda6-4ac3a766c8c1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:41:46.559685 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.559667    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "31c0d1db-215b-4ead-bda6-4ac3a766c8c1" (UID: "31c0d1db-215b-4ead-bda6-4ac3a766c8c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:41:46.559839 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.559731    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-kube-api-access-d4rxb" (OuterVolumeSpecName: "kube-api-access-d4rxb") pod "31c0d1db-215b-4ead-bda6-4ac3a766c8c1" (UID: "31c0d1db-215b-4ead-bda6-4ac3a766c8c1"). InnerVolumeSpecName "kube-api-access-d4rxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:41:46.659295 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.659263    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4rxb\" (UniqueName: \"kubernetes.io/projected/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-kube-api-access-d4rxb\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:41:46.659295 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.659288    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-dshm\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:41:46.659487 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.659303    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-tls-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:41:46.659487 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:46.659325    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31c0d1db-215b-4ead-bda6-4ac3a766c8c1-home\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:41:47.146622 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:47.146593    2566 generic.go:356] "Generic (PLEG): container finished" podID="31c0d1db-215b-4ead-bda6-4ac3a766c8c1" containerID="b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233" exitCode=0
Jul 29 19:41:47.146783 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:47.146636    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"
Jul 29 19:41:47.146783 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:47.146655    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q" event={"ID":"31c0d1db-215b-4ead-bda6-4ac3a766c8c1","Type":"ContainerDied","Data":"b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233"}
Jul 29 19:41:47.146783 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:47.146685    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q" event={"ID":"31c0d1db-215b-4ead-bda6-4ac3a766c8c1","Type":"ContainerDied","Data":"da2867d7c6e843bd605c813ab53eea042e38e62a5b44863c96af42cf3a6804a5"}
Jul 29 19:41:47.146783 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:47.146701    2566 scope.go:117] "RemoveContainer" containerID="b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233"
Jul 29 19:41:47.155359 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:47.155331    2566 scope.go:117] "RemoveContainer" containerID="b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233"
Jul 29 19:41:47.155648 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:41:47.155630    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233\": container with ID starting with b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233 not found: ID does not exist" containerID="b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233"
Jul 29 19:41:47.155713 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:47.155659    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233"} err="failed to get container status \"b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233\": rpc error: code = NotFound desc = could not find container \"b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233\": container with ID starting with b7efebc445fcf3867c4390d814bf6bc55cbaea0bd73e1bdb0c3077fe1c689233 not found: ID does not exist"
Jul 29 19:41:47.170641 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:47.170614    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"]
Jul 29 19:41:47.176611 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:47.176592    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-569c3022-kserve-54944f54c5-22b6q"]
Jul 29 19:41:49.102594 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:41:49.102564    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31c0d1db-215b-4ead-bda6-4ac3a766c8c1" path="/var/lib/kubelet/pods/31c0d1db-215b-4ead-bda6-4ac3a766c8c1/volumes"
Jul 29 19:45:22.030132 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.030102    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"]
Jul 29 19:45:22.030601 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.030562    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31c0d1db-215b-4ead-bda6-4ac3a766c8c1" containerName="main"
Jul 29 19:45:22.030601 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.030579    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="31c0d1db-215b-4ead-bda6-4ac3a766c8c1" containerName="main"
Jul 29 19:45:22.030678 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.030668    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="31c0d1db-215b-4ead-bda6-4ac3a766c8c1" containerName="main"
Jul 29 19:45:22.034040 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.034025    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:22.036895 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.036871    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e9d45d7a\""
Jul 29 19:45:22.045771 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.045738    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"]
Jul 29 19:45:22.067327 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.067300    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"]
Jul 29 19:45:22.070970 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.070954    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:22.073859 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.073841    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e9d45d7a-dockercfg-4wgcq\""
Jul 29 19:45:22.080011 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.079987    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"]
Jul 29 19:45:22.151515 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.151484    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtt2j\" (UniqueName: \"kubernetes.io/projected/7a2175d4-5676-4cb8-9c62-082ab384a2d0-kube-api-access-dtt2j\") pod \"payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5\" (UID: \"7a2175d4-5676-4cb8-9c62-082ab384a2d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:22.151676 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.151552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2175d4-5676-4cb8-9c62-082ab384a2d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5\" (UID: \"7a2175d4-5676-4cb8-9c62-082ab384a2d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:22.252534 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.252504    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f54a71ad-5677-48bf-981b-b79b9f78c572-plugins-config-volume\") pod \"payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl\" (UID: \"f54a71ad-5677-48bf-981b-b79b9f78c572\") " pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:22.252712 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.252574    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtt2j\" (UniqueName: \"kubernetes.io/projected/7a2175d4-5676-4cb8-9c62-082ab384a2d0-kube-api-access-dtt2j\") pod \"payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5\" (UID: \"7a2175d4-5676-4cb8-9c62-082ab384a2d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:22.252712 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.252697    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2175d4-5676-4cb8-9c62-082ab384a2d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5\" (UID: \"7a2175d4-5676-4cb8-9c62-082ab384a2d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:22.252821 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.252769    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sgqg\" (UniqueName: \"kubernetes.io/projected/f54a71ad-5677-48bf-981b-b79b9f78c572-kube-api-access-6sgqg\") pod \"payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl\" (UID: \"f54a71ad-5677-48bf-981b-b79b9f78c572\") " pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:22.253557 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.253528    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2175d4-5676-4cb8-9c62-082ab384a2d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5\" (UID: \"7a2175d4-5676-4cb8-9c62-082ab384a2d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:22.261572 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.261552    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtt2j\" (UniqueName: \"kubernetes.io/projected/7a2175d4-5676-4cb8-9c62-082ab384a2d0-kube-api-access-dtt2j\") pod \"payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5\" (UID: \"7a2175d4-5676-4cb8-9c62-082ab384a2d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:22.346416 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.346283    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:22.353992 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.353965    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f54a71ad-5677-48bf-981b-b79b9f78c572-plugins-config-volume\") pod \"payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl\" (UID: \"f54a71ad-5677-48bf-981b-b79b9f78c572\") " pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:22.354126 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.354024    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sgqg\" (UniqueName: \"kubernetes.io/projected/f54a71ad-5677-48bf-981b-b79b9f78c572-kube-api-access-6sgqg\") pod \"payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl\" (UID: \"f54a71ad-5677-48bf-981b-b79b9f78c572\") " pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:22.354983 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.354962    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f54a71ad-5677-48bf-981b-b79b9f78c572-plugins-config-volume\") pod \"payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl\" (UID: \"f54a71ad-5677-48bf-981b-b79b9f78c572\") " pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:22.362579 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.362558    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sgqg\" (UniqueName: \"kubernetes.io/projected/f54a71ad-5677-48bf-981b-b79b9f78c572-kube-api-access-6sgqg\") pod \"payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl\" (UID: \"f54a71ad-5677-48bf-981b-b79b9f78c572\") " pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:22.381535 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.381501    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:22.501118 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.501083    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"]
Jul 29 19:45:22.506339 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:45:22.506298    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2175d4_5676_4cb8_9c62_082ab384a2d0.slice/crio-3e0afd0dfb22e97dbd8b0fbfa329179551555e5e5b030b3bfc05053bddf14e98 WatchSource:0}: Error finding container 3e0afd0dfb22e97dbd8b0fbfa329179551555e5e5b030b3bfc05053bddf14e98: Status 404 returned error can't find the container with id 3e0afd0dfb22e97dbd8b0fbfa329179551555e5e5b030b3bfc05053bddf14e98
Jul 29 19:45:22.535783 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:22.535691    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"]
Jul 29 19:45:22.538303 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:45:22.537562    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf54a71ad_5677_48bf_981b_b79b9f78c572.slice/crio-6ba78580b1cac351eac93a351a3e3de9ab57657fc7538eec1c0819abee62070f WatchSource:0}: Error finding container 6ba78580b1cac351eac93a351a3e3de9ab57657fc7538eec1c0819abee62070f: Status 404 returned error can't find the container with id 6ba78580b1cac351eac93a351a3e3de9ab57657fc7538eec1c0819abee62070f
Jul 29 19:45:23.173204 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:23.173164    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl" event={"ID":"f54a71ad-5677-48bf-981b-b79b9f78c572","Type":"ContainerStarted","Data":"059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8"}
Jul 29 19:45:23.173661 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:23.173222    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl" event={"ID":"f54a71ad-5677-48bf-981b-b79b9f78c572","Type":"ContainerStarted","Data":"6ba78580b1cac351eac93a351a3e3de9ab57657fc7538eec1c0819abee62070f"}
Jul 29 19:45:23.173661 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:23.173248    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:23.174924 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:23.174892    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5" event={"ID":"7a2175d4-5676-4cb8-9c62-082ab384a2d0","Type":"ContainerStarted","Data":"525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c"}
Jul 29 19:45:23.175034 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:23.174938    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:23.175034 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:23.174954    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5" event={"ID":"7a2175d4-5676-4cb8-9c62-082ab384a2d0","Type":"ContainerStarted","Data":"3e0afd0dfb22e97dbd8b0fbfa329179551555e5e5b030b3bfc05053bddf14e98"}
Jul 29 19:45:23.193118 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:23.193075    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl" podStartSLOduration=1.193059645 podStartE2EDuration="1.193059645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:45:23.191227535 +0000 UTC m=+1688.686988875" watchObservedRunningTime="2026-07-29 19:45:23.193059645 +0000 UTC m=+1688.688820995"
Jul 29 19:45:23.209258 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:23.209211    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5" podStartSLOduration=1.209195383 podStartE2EDuration="1.209195383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:45:23.207746614 +0000 UTC m=+1688.703507955" watchObservedRunningTime="2026-07-29 19:45:23.209195383 +0000 UTC m=+1688.704956724"
Jul 29 19:45:34.182640 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:34.182609    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:34.183092 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:34.182762    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:51.947141 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:51.947104    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"]
Jul 29 19:45:51.947732 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:51.947360    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl" podUID="f54a71ad-5677-48bf-981b-b79b9f78c572" containerName="payload-processing" containerID="cri-o://059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8" gracePeriod=30
Jul 29 19:45:51.952773 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:51.952743    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"]
Jul 29 19:45:51.953006 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:51.952965    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5" podUID="7a2175d4-5676-4cb8-9c62-082ab384a2d0" containerName="payload-pre-processing" containerID="cri-o://525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c" gracePeriod=30
Jul 29 19:45:52.202515 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.202007    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:52.228325 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.228165    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:52.315567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.315528    2566 generic.go:356] "Generic (PLEG): container finished" podID="f54a71ad-5677-48bf-981b-b79b9f78c572" containerID="059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8" exitCode=0
Jul 29 19:45:52.315750 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.315575    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"
Jul 29 19:45:52.315750 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.315612    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl" event={"ID":"f54a71ad-5677-48bf-981b-b79b9f78c572","Type":"ContainerDied","Data":"059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8"}
Jul 29 19:45:52.315750 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.315661    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl" event={"ID":"f54a71ad-5677-48bf-981b-b79b9f78c572","Type":"ContainerDied","Data":"6ba78580b1cac351eac93a351a3e3de9ab57657fc7538eec1c0819abee62070f"}
Jul 29 19:45:52.315750 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.315684    2566 scope.go:117] "RemoveContainer" containerID="059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8"
Jul 29 19:45:52.316836 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.316810    2566 generic.go:356] "Generic (PLEG): container finished" podID="7a2175d4-5676-4cb8-9c62-082ab384a2d0" containerID="525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c" exitCode=0
Jul 29 19:45:52.316918 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.316860    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"
Jul 29 19:45:52.316918 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.316878    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5" event={"ID":"7a2175d4-5676-4cb8-9c62-082ab384a2d0","Type":"ContainerDied","Data":"525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c"}
Jul 29 19:45:52.316995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.316922    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5" event={"ID":"7a2175d4-5676-4cb8-9c62-082ab384a2d0","Type":"ContainerDied","Data":"3e0afd0dfb22e97dbd8b0fbfa329179551555e5e5b030b3bfc05053bddf14e98"}
Jul 29 19:45:52.324308 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.324291    2566 scope.go:117] "RemoveContainer" containerID="059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8"
Jul 29 19:45:52.324583 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:45:52.324562    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8\": container with ID starting with 059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8 not found: ID does not exist" containerID="059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8"
Jul 29 19:45:52.324690 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.324589    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8"} err="failed to get container status \"059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8\": rpc error: code = NotFound desc = could not find container \"059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8\": container with ID starting with 059a378f89b0a95b9a5086dab29260871c86f849ac38a2c26bef652ff7f19ef8 not found: ID does not exist"
Jul 29 19:45:52.324690 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.324606    2566 scope.go:117] "RemoveContainer" containerID="525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c"
Jul 29 19:45:52.330820 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.330802    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f54a71ad-5677-48bf-981b-b79b9f78c572-plugins-config-volume\") pod \"f54a71ad-5677-48bf-981b-b79b9f78c572\" (UID: \"f54a71ad-5677-48bf-981b-b79b9f78c572\") "
Jul 29 19:45:52.330907 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.330864    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sgqg\" (UniqueName: \"kubernetes.io/projected/f54a71ad-5677-48bf-981b-b79b9f78c572-kube-api-access-6sgqg\") pod \"f54a71ad-5677-48bf-981b-b79b9f78c572\" (UID: \"f54a71ad-5677-48bf-981b-b79b9f78c572\") "
Jul 29 19:45:52.330959 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.330918    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2175d4-5676-4cb8-9c62-082ab384a2d0-plugins-config-volume\") pod \"7a2175d4-5676-4cb8-9c62-082ab384a2d0\" (UID: \"7a2175d4-5676-4cb8-9c62-082ab384a2d0\") "
Jul 29 19:45:52.330959 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.330940    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtt2j\" (UniqueName: \"kubernetes.io/projected/7a2175d4-5676-4cb8-9c62-082ab384a2d0-kube-api-access-dtt2j\") pod \"7a2175d4-5676-4cb8-9c62-082ab384a2d0\" (UID: \"7a2175d4-5676-4cb8-9c62-082ab384a2d0\") "
Jul 29 19:45:52.331249 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.331219    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f54a71ad-5677-48bf-981b-b79b9f78c572-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f54a71ad-5677-48bf-981b-b79b9f78c572" (UID: "f54a71ad-5677-48bf-981b-b79b9f78c572"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:45:52.331343 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.331278    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a2175d4-5676-4cb8-9c62-082ab384a2d0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a2175d4-5676-4cb8-9c62-082ab384a2d0" (UID: "7a2175d4-5676-4cb8-9c62-082ab384a2d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:45:52.332163 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.332144    2566 scope.go:117] "RemoveContainer" containerID="525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c"
Jul 29 19:45:52.332438 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:45:52.332418    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c\": container with ID starting with 525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c not found: ID does not exist" containerID="525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c"
Jul 29 19:45:52.332526 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.332445    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c"} err="failed to get container status \"525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c\": rpc error: code = NotFound desc = could not find container \"525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c\": container with ID starting with 525d65aa2e05d1c9c1c9a55775c44ceffee698a53fbc6ca48340b9cb5fba7b9c not found: ID does not exist"
Jul 29 19:45:52.333065 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.333044    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f54a71ad-5677-48bf-981b-b79b9f78c572-kube-api-access-6sgqg" (OuterVolumeSpecName: "kube-api-access-6sgqg") pod "f54a71ad-5677-48bf-981b-b79b9f78c572" (UID: "f54a71ad-5677-48bf-981b-b79b9f78c572"). InnerVolumeSpecName "kube-api-access-6sgqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:45:52.333123 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.333056    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a2175d4-5676-4cb8-9c62-082ab384a2d0-kube-api-access-dtt2j" (OuterVolumeSpecName: "kube-api-access-dtt2j") pod "7a2175d4-5676-4cb8-9c62-082ab384a2d0" (UID: "7a2175d4-5676-4cb8-9c62-082ab384a2d0"). InnerVolumeSpecName "kube-api-access-dtt2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:45:52.432130 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.432091    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f54a71ad-5677-48bf-981b-b79b9f78c572-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:45:52.432130 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.432122    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sgqg\" (UniqueName: \"kubernetes.io/projected/f54a71ad-5677-48bf-981b-b79b9f78c572-kube-api-access-6sgqg\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:45:52.432130 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.432136    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a2175d4-5676-4cb8-9c62-082ab384a2d0-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:45:52.432354 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.432150    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtt2j\" (UniqueName: \"kubernetes.io/projected/7a2175d4-5676-4cb8-9c62-082ab384a2d0-kube-api-access-dtt2j\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:45:52.641302 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.641269    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"]
Jul 29 19:45:52.643082 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.643054    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e9d45d7a-569fd69ff6-z8lz5"]
Jul 29 19:45:52.655698 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.655674    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"]
Jul 29 19:45:52.662794 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:52.662768    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9d45d7a-7b8d7487fd-97bdl"]
Jul 29 19:45:53.102243 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.102210    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a2175d4-5676-4cb8-9c62-082ab384a2d0" path="/var/lib/kubelet/pods/7a2175d4-5676-4cb8-9c62-082ab384a2d0/volumes"
Jul 29 19:45:53.102613 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.102595    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f54a71ad-5677-48bf-981b-b79b9f78c572" path="/var/lib/kubelet/pods/f54a71ad-5677-48bf-981b-b79b9f78c572/volumes"
Jul 29 19:45:53.223902 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.223825    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"]
Jul 29 19:45:53.224240 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.224227    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a2175d4-5676-4cb8-9c62-082ab384a2d0" containerName="payload-pre-processing"
Jul 29 19:45:53.224292 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.224241    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a2175d4-5676-4cb8-9c62-082ab384a2d0" containerName="payload-pre-processing"
Jul 29 19:45:53.224292 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.224254    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f54a71ad-5677-48bf-981b-b79b9f78c572" containerName="payload-processing"
Jul 29 19:45:53.224292 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.224260    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f54a71ad-5677-48bf-981b-b79b9f78c572" containerName="payload-processing"
Jul 29 19:45:53.224400 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.224321    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a2175d4-5676-4cb8-9c62-082ab384a2d0" containerName="payload-pre-processing"
Jul 29 19:45:53.224400 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.224333    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f54a71ad-5677-48bf-981b-b79b9f78c572" containerName="payload-processing"
Jul 29 19:45:53.228705 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.228689    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.231329 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.231303    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-9a86149a-gw-openshift-default-dockercfg-xhzcj\""
Jul 29 19:45:53.231329 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.231321    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 19:45:53.241940 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.241916    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"]
Jul 29 19:45:53.341796 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.341764    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-data\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.341969 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.341813    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-certs\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.341969 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.341872    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-podinfo\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.341969 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.341933    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-credential-socket\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.341969 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.341967    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-token\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.342113 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.342001    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istiod-ca-cert\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.342113 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.342028    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68bsl\" (UniqueName: \"kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-kube-api-access-68bsl\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.342113 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.342081    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-socket\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.342210 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.342114    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-envoy\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.443216 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.443161    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-data\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.443432 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.443238    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-certs\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.443432 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.443288    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-podinfo\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.443432 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.443338    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-credential-socket\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.443432 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.443367    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-token\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.443432 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.443417    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istiod-ca-cert\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.443766 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.443490    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68bsl\" (UniqueName: \"kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-kube-api-access-68bsl\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.443766 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.443529    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-socket\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.443766 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.443677    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-certs\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.443973 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.443849    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-envoy\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.443973 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.443929    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-credential-socket\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.444318 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.444291    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-data\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.444490 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.444370    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-socket\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.444490 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.444404    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istiod-ca-cert\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.446013 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.445979    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-envoy\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.446125 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.446026    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-podinfo\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.453188 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.453162    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-token\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.455431 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.455410    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68bsl\" (UniqueName: \"kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-kube-api-access-68bsl\") pod \"e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") " pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.539238 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.539152    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:53.676673 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.676638    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"]
Jul 29 19:45:53.680088 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:45:53.680062    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d459b53_0473_4ec0_bf6b_c95cb1e146a7.slice/crio-6e3016cb0e8493ab8cba60f63433b9bf1cb6583ec978f711a7aa0300a48e38a7 WatchSource:0}: Error finding container 6e3016cb0e8493ab8cba60f63433b9bf1cb6583ec978f711a7aa0300a48e38a7: Status 404 returned error can't find the container with id 6e3016cb0e8493ab8cba60f63433b9bf1cb6583ec978f711a7aa0300a48e38a7
Jul 29 19:45:53.682074 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:53.682059    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 19:45:54.329520 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:54.329485    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" event={"ID":"7d459b53-0473-4ec0-bf6b-c95cb1e146a7","Type":"ContainerStarted","Data":"6e3016cb0e8493ab8cba60f63433b9bf1cb6583ec978f711a7aa0300a48e38a7"}
Jul 29 19:45:57.024748 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:57.024709    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:45:57.025021 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:57.024789    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:45:57.025021 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:57.024818    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:45:57.348897 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:57.348802    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" event={"ID":"7d459b53-0473-4ec0-bf6b-c95cb1e146a7","Type":"ContainerStarted","Data":"85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589"}
Jul 29 19:45:57.371821 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:57.371763    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" podStartSLOduration=1.029572872 podStartE2EDuration="4.371742002s" totalImagesPullingTime="3.34216913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:45:57.370244968 +0000 UTC m=+1722.866006333" watchObservedRunningTime="2026-07-29 19:45:57.371742002 +0000 UTC m=+1722.867503345"
Jul 29 19:45:57.539538 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:57.539509    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:58.545690 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:58.545661    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:59.359796 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.359762    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:59.360855 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.360834    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:45:59.578765 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.578733    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"]
Jul 29 19:45:59.582286 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.582269    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:45:59.585618 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.585594    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-9a86149a\""
Jul 29 19:45:59.594538 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.594491    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"]
Jul 29 19:45:59.597339 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.597317    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"]
Jul 29 19:45:59.600746 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.600726    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:45:59.603275 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.603257    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-9a86149a-dockercfg-g72lx\""
Jul 29 19:45:59.611530 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.611425    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"]
Jul 29 19:45:59.708828 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.708794    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58dbh\" (UniqueName: \"kubernetes.io/projected/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-kube-api-access-58dbh\") pod \"payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49\" (UID: \"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0\") " pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:45:59.708984 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.708836    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkpfp\" (UniqueName: \"kubernetes.io/projected/e0824f49-6099-48d0-a7e3-a9892e2825f2-kube-api-access-wkpfp\") pod \"payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j\" (UID: \"e0824f49-6099-48d0-a7e3-a9892e2825f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:45:59.708984 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.708945    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49\" (UID: \"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0\") " pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:45:59.709058 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.709008    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0824f49-6099-48d0-a7e3-a9892e2825f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j\" (UID: \"e0824f49-6099-48d0-a7e3-a9892e2825f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:45:59.810249 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.810208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58dbh\" (UniqueName: \"kubernetes.io/projected/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-kube-api-access-58dbh\") pod \"payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49\" (UID: \"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0\") " pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:45:59.810423 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.810358    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkpfp\" (UniqueName: \"kubernetes.io/projected/e0824f49-6099-48d0-a7e3-a9892e2825f2-kube-api-access-wkpfp\") pod \"payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j\" (UID: \"e0824f49-6099-48d0-a7e3-a9892e2825f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:45:59.810496 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.810472    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49\" (UID: \"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0\") " pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:45:59.810540 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.810526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0824f49-6099-48d0-a7e3-a9892e2825f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j\" (UID: \"e0824f49-6099-48d0-a7e3-a9892e2825f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:45:59.811212 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.811195    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0824f49-6099-48d0-a7e3-a9892e2825f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j\" (UID: \"e0824f49-6099-48d0-a7e3-a9892e2825f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:45:59.811268 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.811207    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49\" (UID: \"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0\") " pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:45:59.819509 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.819480    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58dbh\" (UniqueName: \"kubernetes.io/projected/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-kube-api-access-58dbh\") pod \"payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49\" (UID: \"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0\") " pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:45:59.819635 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.819621    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkpfp\" (UniqueName: \"kubernetes.io/projected/e0824f49-6099-48d0-a7e3-a9892e2825f2-kube-api-access-wkpfp\") pod \"payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j\" (UID: \"e0824f49-6099-48d0-a7e3-a9892e2825f2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:45:59.893595 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.893514    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:45:59.912327 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:45:59.912304    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:46:00.075423 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:00.075402    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"]
Jul 29 19:46:00.077251 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:46:00.077218    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod384cbbf4_a4d8_45bf_aef1_e3c378d2eda0.slice/crio-6094cac562a4fa85c7a3a0a04b0e9bd7a68d1590cb98ac9b6cf1b6b7cc3f4a20 WatchSource:0}: Error finding container 6094cac562a4fa85c7a3a0a04b0e9bd7a68d1590cb98ac9b6cf1b6b7cc3f4a20: Status 404 returned error can't find the container with id 6094cac562a4fa85c7a3a0a04b0e9bd7a68d1590cb98ac9b6cf1b6b7cc3f4a20
Jul 29 19:46:00.255567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:00.255537    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"]
Jul 29 19:46:00.257277 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:46:00.257252    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0824f49_6099_48d0_a7e3_a9892e2825f2.slice/crio-c04fe18c63f345dc39b57997cc40b2dc7bfc059c5de74b45bcf3bf91e138123d WatchSource:0}: Error finding container c04fe18c63f345dc39b57997cc40b2dc7bfc059c5de74b45bcf3bf91e138123d: Status 404 returned error can't find the container with id c04fe18c63f345dc39b57997cc40b2dc7bfc059c5de74b45bcf3bf91e138123d
Jul 29 19:46:00.366967 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:00.366932    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j" event={"ID":"e0824f49-6099-48d0-a7e3-a9892e2825f2","Type":"ContainerStarted","Data":"4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219"}
Jul 29 19:46:00.367156 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:00.366985    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j" event={"ID":"e0824f49-6099-48d0-a7e3-a9892e2825f2","Type":"ContainerStarted","Data":"c04fe18c63f345dc39b57997cc40b2dc7bfc059c5de74b45bcf3bf91e138123d"}
Jul 29 19:46:00.367156 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:00.367098    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:46:00.368556 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:00.368528    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49" event={"ID":"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0","Type":"ContainerStarted","Data":"39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282"}
Jul 29 19:46:00.368670 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:00.368576    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:46:00.368670 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:00.368591    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49" event={"ID":"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0","Type":"ContainerStarted","Data":"6094cac562a4fa85c7a3a0a04b0e9bd7a68d1590cb98ac9b6cf1b6b7cc3f4a20"}
Jul 29 19:46:00.386156 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:00.386117    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j" podStartSLOduration=1.3861029710000001 podStartE2EDuration="1.386102971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:45:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:46:00.383620959 +0000 UTC m=+1725.879382302" watchObservedRunningTime="2026-07-29 19:46:00.386102971 +0000 UTC m=+1725.881864351"
Jul 29 19:46:00.403991 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:00.403942    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49" podStartSLOduration=1.403929528 podStartE2EDuration="1.403929528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:45:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:46:00.402020571 +0000 UTC m=+1725.897781943" watchObservedRunningTime="2026-07-29 19:46:00.403929528 +0000 UTC m=+1725.899690870"
Jul 29 19:46:11.376562 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:11.376531    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:46:11.377046 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:11.376603    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:46:11.428861 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:11.428833    2566 generic.go:356] "Generic (PLEG): container finished" podID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerID="85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589" exitCode=0
Jul 29 19:46:11.429043 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:11.428905    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" event={"ID":"7d459b53-0473-4ec0-bf6b-c95cb1e146a7","Type":"ContainerDied","Data":"85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589"}
Jul 29 19:46:11.429277 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:11.429261    2566 scope.go:117] "RemoveContainer" containerID="85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589"
Jul 29 19:46:11.431223 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:11.431194    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:46:11.431347 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:11.431261    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:46:11.431347 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:11.431294    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:46:12.435491 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:12.435419    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" event={"ID":"7d459b53-0473-4ec0-bf6b-c95cb1e146a7","Type":"ContainerStarted","Data":"e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf"}
Jul 29 19:46:12.540564 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:12.540522    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:46:12.541665 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:12.541635    2566 patch_prober.go:28] interesting pod/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 19:46:12.541780 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:12.541699    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 19:46:13.539843 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:13.539804    2566 patch_prober.go:28] interesting pod/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 19:46:13.540329 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:13.539865    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 19:46:14.360476 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:14.359693    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:46:14.539862 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:14.539819    2566 patch_prober.go:28] interesting pod/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 19:46:14.540331 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:14.539896    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 19:46:15.540817 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:15.540335    2566 patch_prober.go:28] interesting pod/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 29 19:46:15.540817 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:15.540395    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 29 19:46:16.544149 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:16.544119    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:46:16.545213 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:16.545191    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:46:30.209202 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.209166    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"]
Jul 29 19:46:30.209790 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.209414    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49" podUID="384cbbf4-a4d8-45bf-aef1-e3c378d2eda0" containerName="payload-processing" containerID="cri-o://39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282" gracePeriod=30
Jul 29 19:46:30.214728 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.214701    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"]
Jul 29 19:46:30.214990 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.214957    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j" podUID="e0824f49-6099-48d0-a7e3-a9892e2825f2" containerName="payload-pre-processing" containerID="cri-o://4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219" gracePeriod=30
Jul 29 19:46:30.504689 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.504648    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"]
Jul 29 19:46:30.504887 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.504867    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy" containerID="cri-o://e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf" gracePeriod=30
Jul 29 19:46:30.506317 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.506291    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:46:30.508619 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.508599    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:46:30.535517 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.535481    2566 generic.go:356] "Generic (PLEG): container finished" podID="e0824f49-6099-48d0-a7e3-a9892e2825f2" containerID="4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219" exitCode=0
Jul 29 19:46:30.535686 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.535571    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j" event={"ID":"e0824f49-6099-48d0-a7e3-a9892e2825f2","Type":"ContainerDied","Data":"4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219"}
Jul 29 19:46:30.535686 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.535582    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"
Jul 29 19:46:30.535686 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.535650    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j" event={"ID":"e0824f49-6099-48d0-a7e3-a9892e2825f2","Type":"ContainerDied","Data":"c04fe18c63f345dc39b57997cc40b2dc7bfc059c5de74b45bcf3bf91e138123d"}
Jul 29 19:46:30.535686 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.535684    2566 scope.go:117] "RemoveContainer" containerID="4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219"
Jul 29 19:46:30.536934 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.536913    2566 generic.go:356] "Generic (PLEG): container finished" podID="384cbbf4-a4d8-45bf-aef1-e3c378d2eda0" containerID="39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282" exitCode=0
Jul 29 19:46:30.537041 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.536954    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"
Jul 29 19:46:30.537098 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.536956    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49" event={"ID":"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0","Type":"ContainerDied","Data":"39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282"}
Jul 29 19:46:30.537138 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.537102    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49" event={"ID":"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0","Type":"ContainerDied","Data":"6094cac562a4fa85c7a3a0a04b0e9bd7a68d1590cb98ac9b6cf1b6b7cc3f4a20"}
Jul 29 19:46:30.544958 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.544911    2566 scope.go:117] "RemoveContainer" containerID="4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219"
Jul 29 19:46:30.545233 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:46:30.545214    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219\": container with ID starting with 4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219 not found: ID does not exist" containerID="4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219"
Jul 29 19:46:30.545329 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.545262    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219"} err="failed to get container status \"4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219\": rpc error: code = NotFound desc = could not find container \"4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219\": container with ID starting with 4385104ea0c27ff626f6652bdb3215cc7f570e953e7547d5954875c4da4d2219 not found: ID does not exist"
Jul 29 19:46:30.545329 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.545288    2566 scope.go:117] "RemoveContainer" containerID="39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282"
Jul 29 19:46:30.554342 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.554323    2566 scope.go:117] "RemoveContainer" containerID="39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282"
Jul 29 19:46:30.554608 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:46:30.554588    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282\": container with ID starting with 39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282 not found: ID does not exist" containerID="39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282"
Jul 29 19:46:30.554672 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.554617    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282"} err="failed to get container status \"39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282\": rpc error: code = NotFound desc = could not find container \"39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282\": container with ID starting with 39f51ec8eef139b5a8a0f29546875f9d0060772cdc0b714336aee10e5a554282 not found: ID does not exist"
Jul 29 19:46:30.623058 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.623029    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0824f49-6099-48d0-a7e3-a9892e2825f2-plugins-config-volume\") pod \"e0824f49-6099-48d0-a7e3-a9892e2825f2\" (UID: \"e0824f49-6099-48d0-a7e3-a9892e2825f2\") "
Jul 29 19:46:30.623251 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.623099    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-plugins-config-volume\") pod \"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0\" (UID: \"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0\") "
Jul 29 19:46:30.623251 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.623150    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkpfp\" (UniqueName: \"kubernetes.io/projected/e0824f49-6099-48d0-a7e3-a9892e2825f2-kube-api-access-wkpfp\") pod \"e0824f49-6099-48d0-a7e3-a9892e2825f2\" (UID: \"e0824f49-6099-48d0-a7e3-a9892e2825f2\") "
Jul 29 19:46:30.623251 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.623194    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58dbh\" (UniqueName: \"kubernetes.io/projected/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-kube-api-access-58dbh\") pod \"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0\" (UID: \"384cbbf4-a4d8-45bf-aef1-e3c378d2eda0\") "
Jul 29 19:46:30.623437 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.623408    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0824f49-6099-48d0-a7e3-a9892e2825f2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e0824f49-6099-48d0-a7e3-a9892e2825f2" (UID: "e0824f49-6099-48d0-a7e3-a9892e2825f2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:46:30.623534 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.623511    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "384cbbf4-a4d8-45bf-aef1-e3c378d2eda0" (UID: "384cbbf4-a4d8-45bf-aef1-e3c378d2eda0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:46:30.623627 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.623612    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:30.623693 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.623630    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0824f49-6099-48d0-a7e3-a9892e2825f2-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:30.625317 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.625283    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-kube-api-access-58dbh" (OuterVolumeSpecName: "kube-api-access-58dbh") pod "384cbbf4-a4d8-45bf-aef1-e3c378d2eda0" (UID: "384cbbf4-a4d8-45bf-aef1-e3c378d2eda0"). InnerVolumeSpecName "kube-api-access-58dbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:46:30.625440 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.625382    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0824f49-6099-48d0-a7e3-a9892e2825f2-kube-api-access-wkpfp" (OuterVolumeSpecName: "kube-api-access-wkpfp") pod "e0824f49-6099-48d0-a7e3-a9892e2825f2" (UID: "e0824f49-6099-48d0-a7e3-a9892e2825f2"). InnerVolumeSpecName "kube-api-access-wkpfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:46:30.724622 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.724591    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkpfp\" (UniqueName: \"kubernetes.io/projected/e0824f49-6099-48d0-a7e3-a9892e2825f2-kube-api-access-wkpfp\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:30.724622 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.724617    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58dbh\" (UniqueName: \"kubernetes.io/projected/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0-kube-api-access-58dbh\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:30.865856 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.865820    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"]
Jul 29 19:46:30.868772 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.868750    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9a86149a-66fd5fd44-7bm4j"]
Jul 29 19:46:30.882966 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.882940    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"]
Jul 29 19:46:30.886551 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:30.886530    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9a86149a-55845dc9fc-m8d49"]
Jul 29 19:46:31.102591 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:31.102507    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="384cbbf4-a4d8-45bf-aef1-e3c378d2eda0" path="/var/lib/kubelet/pods/384cbbf4-a4d8-45bf-aef1-e3c378d2eda0/volumes"
Jul 29 19:46:31.102910 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:31.102898    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0824f49-6099-48d0-a7e3-a9892e2825f2" path="/var/lib/kubelet/pods/e0824f49-6099-48d0-a7e3-a9892e2825f2/volumes"
Jul 29 19:46:31.546424 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:31.546381    2566 patch_prober.go:28] interesting pod/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 29 19:46:31.546932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:31.546443    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 19:46:34.631338 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.631303    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"]
Jul 29 19:46:34.631824 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.631697    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0824f49-6099-48d0-a7e3-a9892e2825f2" containerName="payload-pre-processing"
Jul 29 19:46:34.631824 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.631709    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0824f49-6099-48d0-a7e3-a9892e2825f2" containerName="payload-pre-processing"
Jul 29 19:46:34.631824 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.631721    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="384cbbf4-a4d8-45bf-aef1-e3c378d2eda0" containerName="payload-processing"
Jul 29 19:46:34.631824 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.631728    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="384cbbf4-a4d8-45bf-aef1-e3c378d2eda0" containerName="payload-processing"
Jul 29 19:46:34.631824 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.631809    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0824f49-6099-48d0-a7e3-a9892e2825f2" containerName="payload-pre-processing"
Jul 29 19:46:34.631824 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.631821    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="384cbbf4-a4d8-45bf-aef1-e3c378d2eda0" containerName="payload-processing"
Jul 29 19:46:34.636832 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.636811    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:46:34.639343 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.639316    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-2f25c52f\""
Jul 29 19:46:34.644407 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.644383    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"]
Jul 29 19:46:34.662824 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.662795    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"]
Jul 29 19:46:34.667038 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.667018    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:46:34.669629 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.669609    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-2f25c52f-dockercfg-wq9cj\""
Jul 29 19:46:34.673930 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.673908    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"]
Jul 29 19:46:34.766418 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.766368    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dwm9\" (UniqueName: \"kubernetes.io/projected/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-kube-api-access-6dwm9\") pod \"payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd\" (UID: \"bcd92489-1fc2-4355-a15f-f44bfc2bd84a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:46:34.766595 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.766485    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd\" (UID: \"bcd92489-1fc2-4355-a15f-f44bfc2bd84a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:46:34.766652 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.766626    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26t8w\" (UniqueName: \"kubernetes.io/projected/6f255cfe-910a-413b-9b8c-784f35a05913-kube-api-access-26t8w\") pod \"payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9\" (UID: \"6f255cfe-910a-413b-9b8c-784f35a05913\") " pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:46:34.766696 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.766675    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f255cfe-910a-413b-9b8c-784f35a05913-plugins-config-volume\") pod \"payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9\" (UID: \"6f255cfe-910a-413b-9b8c-784f35a05913\") " pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:46:34.867551 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.867514    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd\" (UID: \"bcd92489-1fc2-4355-a15f-f44bfc2bd84a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:46:34.867739 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.867618    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26t8w\" (UniqueName: \"kubernetes.io/projected/6f255cfe-910a-413b-9b8c-784f35a05913-kube-api-access-26t8w\") pod \"payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9\" (UID: \"6f255cfe-910a-413b-9b8c-784f35a05913\") " pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:46:34.867739 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.867663    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f255cfe-910a-413b-9b8c-784f35a05913-plugins-config-volume\") pod \"payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9\" (UID: \"6f255cfe-910a-413b-9b8c-784f35a05913\") " pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:46:34.867739 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.867710    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dwm9\" (UniqueName: \"kubernetes.io/projected/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-kube-api-access-6dwm9\") pod \"payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd\" (UID: \"bcd92489-1fc2-4355-a15f-f44bfc2bd84a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:46:34.868199 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.868178    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd\" (UID: \"bcd92489-1fc2-4355-a15f-f44bfc2bd84a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:46:34.868304 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.868285    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f255cfe-910a-413b-9b8c-784f35a05913-plugins-config-volume\") pod \"payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9\" (UID: \"6f255cfe-910a-413b-9b8c-784f35a05913\") " pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:46:34.877929 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.877895    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26t8w\" (UniqueName: \"kubernetes.io/projected/6f255cfe-910a-413b-9b8c-784f35a05913-kube-api-access-26t8w\") pod \"payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9\" (UID: \"6f255cfe-910a-413b-9b8c-784f35a05913\") " pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:46:34.878111 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.878090    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dwm9\" (UniqueName: \"kubernetes.io/projected/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-kube-api-access-6dwm9\") pod \"payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd\" (UID: \"bcd92489-1fc2-4355-a15f-f44bfc2bd84a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:46:34.949862 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.949831    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:46:34.978849 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:34.978819    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:46:35.311841 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.311816    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"]
Jul 29 19:46:35.313620 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:46:35.313595    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd92489_1fc2_4355_a15f_f44bfc2bd84a.slice/crio-755869803fdd333e084e0c1f0089d003564f72634c6f2e20f64eb22dba3ac84c WatchSource:0}: Error finding container 755869803fdd333e084e0c1f0089d003564f72634c6f2e20f64eb22dba3ac84c: Status 404 returned error can't find the container with id 755869803fdd333e084e0c1f0089d003564f72634c6f2e20f64eb22dba3ac84c
Jul 29 19:46:35.351187 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.351156    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"]
Jul 29 19:46:35.355539 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:46:35.355507    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f255cfe_910a_413b_9b8c_784f35a05913.slice/crio-2cfb8179410a4d0c66bbd090beb12bd2d4fe5865a1478e2afc3477a58303b648 WatchSource:0}: Error finding container 2cfb8179410a4d0c66bbd090beb12bd2d4fe5865a1478e2afc3477a58303b648: Status 404 returned error can't find the container with id 2cfb8179410a4d0c66bbd090beb12bd2d4fe5865a1478e2afc3477a58303b648
Jul 29 19:46:35.566813 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.566725    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9" event={"ID":"6f255cfe-910a-413b-9b8c-784f35a05913","Type":"ContainerStarted","Data":"e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06"}
Jul 29 19:46:35.566813 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.566780    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9" event={"ID":"6f255cfe-910a-413b-9b8c-784f35a05913","Type":"ContainerStarted","Data":"2cfb8179410a4d0c66bbd090beb12bd2d4fe5865a1478e2afc3477a58303b648"}
Jul 29 19:46:35.567035 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.566840    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:46:35.568328 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.568302    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd" event={"ID":"bcd92489-1fc2-4355-a15f-f44bfc2bd84a","Type":"ContainerStarted","Data":"fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8"}
Jul 29 19:46:35.568437 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.568351    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd" event={"ID":"bcd92489-1fc2-4355-a15f-f44bfc2bd84a","Type":"ContainerStarted","Data":"755869803fdd333e084e0c1f0089d003564f72634c6f2e20f64eb22dba3ac84c"}
Jul 29 19:46:35.568437 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.568374    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:46:35.587837 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.587795    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9" podStartSLOduration=1.587781558 podStartE2EDuration="1.587781558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:46:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:46:35.585656763 +0000 UTC m=+1761.081418110" watchObservedRunningTime="2026-07-29 19:46:35.587781558 +0000 UTC m=+1761.083542900"
Jul 29 19:46:35.603121 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.603078    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd" podStartSLOduration=1.6030648159999998 podStartE2EDuration="1.603064816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:46:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:46:35.60116504 +0000 UTC m=+1761.096926394" watchObservedRunningTime="2026-07-29 19:46:35.603064816 +0000 UTC m=+1761.098826158"
Jul 29 19:46:35.757729 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.757702    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:46:35.879500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879367    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-socket\") pod \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") "
Jul 29 19:46:35.879500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879417    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-certs\") pod \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") "
Jul 29 19:46:35.879500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879443    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-podinfo\") pod \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") "
Jul 29 19:46:35.879500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879495    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istiod-ca-cert\") pod \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") "
Jul 29 19:46:35.879833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879600    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68bsl\" (UniqueName: \"kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-kube-api-access-68bsl\") pod \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") "
Jul 29 19:46:35.879833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879630    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-data\") pod \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") "
Jul 29 19:46:35.879833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879655    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7d459b53-0473-4ec0-bf6b-c95cb1e146a7" (UID: "7d459b53-0473-4ec0-bf6b-c95cb1e146a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:46:35.879833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879681    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-credential-socket\") pod \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") "
Jul 29 19:46:35.879833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879690    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7d459b53-0473-4ec0-bf6b-c95cb1e146a7" (UID: "7d459b53-0473-4ec0-bf6b-c95cb1e146a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:46:35.879833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879707    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-token\") pod \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") "
Jul 29 19:46:35.879833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879732    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-envoy\") pod \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\" (UID: \"7d459b53-0473-4ec0-bf6b-c95cb1e146a7\") "
Jul 29 19:46:35.880202 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.879887    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7d459b53-0473-4ec0-bf6b-c95cb1e146a7" (UID: "7d459b53-0473-4ec0-bf6b-c95cb1e146a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:46:35.880202 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.880040    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:35.880202 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.880056    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-workload-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:35.880202 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.880070    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istiod-ca-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:35.880202 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.880137    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7d459b53-0473-4ec0-bf6b-c95cb1e146a7" (UID: "7d459b53-0473-4ec0-bf6b-c95cb1e146a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:46:35.880476 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.880409    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "7d459b53-0473-4ec0-bf6b-c95cb1e146a7" (UID: "7d459b53-0473-4ec0-bf6b-c95cb1e146a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:46:35.882052 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.882020    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7d459b53-0473-4ec0-bf6b-c95cb1e146a7" (UID: "7d459b53-0473-4ec0-bf6b-c95cb1e146a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 19:46:35.882171 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.882087    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d459b53-0473-4ec0-bf6b-c95cb1e146a7" (UID: "7d459b53-0473-4ec0-bf6b-c95cb1e146a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:46:35.882171 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.882149    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-kube-api-access-68bsl" (OuterVolumeSpecName: "kube-api-access-68bsl") pod "7d459b53-0473-4ec0-bf6b-c95cb1e146a7" (UID: "7d459b53-0473-4ec0-bf6b-c95cb1e146a7"). InnerVolumeSpecName "kube-api-access-68bsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:46:35.882259 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.882222    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7d459b53-0473-4ec0-bf6b-c95cb1e146a7" (UID: "7d459b53-0473-4ec0-bf6b-c95cb1e146a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:46:35.980949 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.980910    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68bsl\" (UniqueName: \"kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-kube-api-access-68bsl\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:35.980949 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.980944    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-data\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:35.980949 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.980954    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-credential-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:35.981183 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.980965    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-token\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:35.981183 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.980974    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-envoy\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:35.981183 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:35.980981    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d459b53-0473-4ec0-bf6b-c95cb1e146a7-istio-podinfo\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:46:36.574803 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.574763    2566 generic.go:356] "Generic (PLEG): container finished" podID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerID="e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf" exitCode=0
Jul 29 19:46:36.575005 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.574818    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"
Jul 29 19:46:36.575005 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.574851    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" event={"ID":"7d459b53-0473-4ec0-bf6b-c95cb1e146a7","Type":"ContainerDied","Data":"e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf"}
Jul 29 19:46:36.575005 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.574913    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq" event={"ID":"7d459b53-0473-4ec0-bf6b-c95cb1e146a7","Type":"ContainerDied","Data":"6e3016cb0e8493ab8cba60f63433b9bf1cb6583ec978f711a7aa0300a48e38a7"}
Jul 29 19:46:36.575005 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.574947    2566 scope.go:117] "RemoveContainer" containerID="e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf"
Jul 29 19:46:36.585212 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.585194    2566 scope.go:117] "RemoveContainer" containerID="85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589"
Jul 29 19:46:36.593476 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.593437    2566 scope.go:117] "RemoveContainer" containerID="e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf"
Jul 29 19:46:36.593688 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:46:36.593670    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf\": container with ID starting with e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf not found: ID does not exist" containerID="e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf"
Jul 29 19:46:36.593736 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.593697    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf"} err="failed to get container status \"e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf\": rpc error: code = NotFound desc = could not find container \"e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf\": container with ID starting with e4df728ca34da571b8ec7c3fa65b3fd00dfc15e8b96c1c5d236f9592738da4cf not found: ID does not exist"
Jul 29 19:46:36.593736 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.593713    2566 scope.go:117] "RemoveContainer" containerID="85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589"
Jul 29 19:46:36.593936 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:46:36.593920    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589\": container with ID starting with 85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589 not found: ID does not exist" containerID="85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589"
Jul 29 19:46:36.593984 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.593944    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589"} err="failed to get container status \"85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589\": rpc error: code = NotFound desc = could not find container \"85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589\": container with ID starting with 85976cd138005d044a3b9f1bf1862a7938a9bf248f5c48628d346c3c7661e589 not found: ID does not exist"
Jul 29 19:46:36.598886 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.598867    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"]
Jul 29 19:46:36.600618 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:36.600599    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-9a86149a-gw-openshift-default-6458b4cb8-z6kcq"]
Jul 29 19:46:37.101782 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:37.101752    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" path="/var/lib/kubelet/pods/7d459b53-0473-4ec0-bf6b-c95cb1e146a7/volumes"
Jul 29 19:46:46.577356 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:46.577324    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:46:46.577906 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:46:46.577626    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:47:04.388257 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.388213    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"]
Jul 29 19:47:04.388786 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.388438    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9" podUID="6f255cfe-910a-413b-9b8c-784f35a05913" containerName="payload-processing" containerID="cri-o://e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06" gracePeriod=30
Jul 29 19:47:04.399586 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.399555    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"]
Jul 29 19:47:04.399816 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.399780    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd" podUID="bcd92489-1fc2-4355-a15f-f44bfc2bd84a" containerName="payload-pre-processing" containerID="cri-o://fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8" gracePeriod=30
Jul 29 19:47:04.646129 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.646049    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:47:04.672100 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.672075    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:47:04.716565 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.716532    2566 generic.go:356] "Generic (PLEG): container finished" podID="6f255cfe-910a-413b-9b8c-784f35a05913" containerID="e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06" exitCode=0
Jul 29 19:47:04.716745 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.716592    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"
Jul 29 19:47:04.716745 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.716613    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9" event={"ID":"6f255cfe-910a-413b-9b8c-784f35a05913","Type":"ContainerDied","Data":"e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06"}
Jul 29 19:47:04.716745 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.716708    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9" event={"ID":"6f255cfe-910a-413b-9b8c-784f35a05913","Type":"ContainerDied","Data":"2cfb8179410a4d0c66bbd090beb12bd2d4fe5865a1478e2afc3477a58303b648"}
Jul 29 19:47:04.716745 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.716739    2566 scope.go:117] "RemoveContainer" containerID="e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06"
Jul 29 19:47:04.717931 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.717909    2566 generic.go:356] "Generic (PLEG): container finished" podID="bcd92489-1fc2-4355-a15f-f44bfc2bd84a" containerID="fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8" exitCode=0
Jul 29 19:47:04.718037 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.717946    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd" event={"ID":"bcd92489-1fc2-4355-a15f-f44bfc2bd84a","Type":"ContainerDied","Data":"fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8"}
Jul 29 19:47:04.718037 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.717975    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"
Jul 29 19:47:04.718037 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.717994    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd" event={"ID":"bcd92489-1fc2-4355-a15f-f44bfc2bd84a","Type":"ContainerDied","Data":"755869803fdd333e084e0c1f0089d003564f72634c6f2e20f64eb22dba3ac84c"}
Jul 29 19:47:04.724854 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.724838    2566 scope.go:117] "RemoveContainer" containerID="e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06"
Jul 29 19:47:04.725091 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:47:04.725073    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06\": container with ID starting with e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06 not found: ID does not exist" containerID="e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06"
Jul 29 19:47:04.725160 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.725096    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06"} err="failed to get container status \"e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06\": rpc error: code = NotFound desc = could not find container \"e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06\": container with ID starting with e396eb779db50f9ac75ef280dbfc023952f3b1752988571ef7da85b35b4e5f06 not found: ID does not exist"
Jul 29 19:47:04.725160 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.725110    2566 scope.go:117] "RemoveContainer" containerID="fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8"
Jul 29 19:47:04.732258 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.732239    2566 scope.go:117] "RemoveContainer" containerID="fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8"
Jul 29 19:47:04.732483 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:47:04.732460    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8\": container with ID starting with fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8 not found: ID does not exist" containerID="fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8"
Jul 29 19:47:04.732532 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.732490    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8"} err="failed to get container status \"fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8\": rpc error: code = NotFound desc = could not find container \"fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8\": container with ID starting with fca3bfa5a4c7bc7a826bea3ba84067ff4077beac14a4f19fa138c85ceff7f9a8 not found: ID does not exist"
Jul 29 19:47:04.745185 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.745163    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f255cfe-910a-413b-9b8c-784f35a05913-plugins-config-volume\") pod \"6f255cfe-910a-413b-9b8c-784f35a05913\" (UID: \"6f255cfe-910a-413b-9b8c-784f35a05913\") "
Jul 29 19:47:04.745269 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.745209    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-plugins-config-volume\") pod \"bcd92489-1fc2-4355-a15f-f44bfc2bd84a\" (UID: \"bcd92489-1fc2-4355-a15f-f44bfc2bd84a\") "
Jul 29 19:47:04.745309 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.745280    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26t8w\" (UniqueName: \"kubernetes.io/projected/6f255cfe-910a-413b-9b8c-784f35a05913-kube-api-access-26t8w\") pod \"6f255cfe-910a-413b-9b8c-784f35a05913\" (UID: \"6f255cfe-910a-413b-9b8c-784f35a05913\") "
Jul 29 19:47:04.745355 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.745307    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dwm9\" (UniqueName: \"kubernetes.io/projected/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-kube-api-access-6dwm9\") pod \"bcd92489-1fc2-4355-a15f-f44bfc2bd84a\" (UID: \"bcd92489-1fc2-4355-a15f-f44bfc2bd84a\") "
Jul 29 19:47:04.745638 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.745598    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f255cfe-910a-413b-9b8c-784f35a05913-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6f255cfe-910a-413b-9b8c-784f35a05913" (UID: "6f255cfe-910a-413b-9b8c-784f35a05913"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:47:04.745638 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.745606    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bcd92489-1fc2-4355-a15f-f44bfc2bd84a" (UID: "bcd92489-1fc2-4355-a15f-f44bfc2bd84a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:47:04.747255 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.747232    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f255cfe-910a-413b-9b8c-784f35a05913-kube-api-access-26t8w" (OuterVolumeSpecName: "kube-api-access-26t8w") pod "6f255cfe-910a-413b-9b8c-784f35a05913" (UID: "6f255cfe-910a-413b-9b8c-784f35a05913"). InnerVolumeSpecName "kube-api-access-26t8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:47:04.747340 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.747267    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-kube-api-access-6dwm9" (OuterVolumeSpecName: "kube-api-access-6dwm9") pod "bcd92489-1fc2-4355-a15f-f44bfc2bd84a" (UID: "bcd92489-1fc2-4355-a15f-f44bfc2bd84a"). InnerVolumeSpecName "kube-api-access-6dwm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:47:04.846404 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.846372    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f255cfe-910a-413b-9b8c-784f35a05913-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:04.846404 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.846401    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:04.846404 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.846410    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26t8w\" (UniqueName: \"kubernetes.io/projected/6f255cfe-910a-413b-9b8c-784f35a05913-kube-api-access-26t8w\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:04.846646 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:04.846419    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dwm9\" (UniqueName: \"kubernetes.io/projected/bcd92489-1fc2-4355-a15f-f44bfc2bd84a-kube-api-access-6dwm9\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:05.046119 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:05.046088    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"]
Jul 29 19:47:05.050748 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:05.050721    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2f25c52f-7c96c64544-bbftd"]
Jul 29 19:47:05.061686 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:05.061665    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"]
Jul 29 19:47:05.065292 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:05.065263    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f25c52f-7c8bcfccfd-p4gj9"]
Jul 29 19:47:05.101869 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:05.101838    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f255cfe-910a-413b-9b8c-784f35a05913" path="/var/lib/kubelet/pods/6f255cfe-910a-413b-9b8c-784f35a05913/volumes"
Jul 29 19:47:05.102201 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:05.102189    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcd92489-1fc2-4355-a15f-f44bfc2bd84a" path="/var/lib/kubelet/pods/bcd92489-1fc2-4355-a15f-f44bfc2bd84a/volumes"
Jul 29 19:47:06.321542 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.321466    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"]
Jul 29 19:47:06.321891 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.321845    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy"
Jul 29 19:47:06.321891 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.321856    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy"
Jul 29 19:47:06.321891 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.321871    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f255cfe-910a-413b-9b8c-784f35a05913" containerName="payload-processing"
Jul 29 19:47:06.321891 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.321877    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f255cfe-910a-413b-9b8c-784f35a05913" containerName="payload-processing"
Jul 29 19:47:06.322023 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.321898    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcd92489-1fc2-4355-a15f-f44bfc2bd84a" containerName="payload-pre-processing"
Jul 29 19:47:06.322023 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.321904    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcd92489-1fc2-4355-a15f-f44bfc2bd84a" containerName="payload-pre-processing"
Jul 29 19:47:06.322023 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.321959    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy"
Jul 29 19:47:06.322023 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.321968    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcd92489-1fc2-4355-a15f-f44bfc2bd84a" containerName="payload-pre-processing"
Jul 29 19:47:06.322023 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.321976    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f255cfe-910a-413b-9b8c-784f35a05913" containerName="payload-processing"
Jul 29 19:47:06.322023 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.321981    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy"
Jul 29 19:47:06.326507 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.326489    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.329673 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.329651    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ac4a75bf-openshift-default-dockercfg-7wdbr\""
Jul 29 19:47:06.329801 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.329766    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 19:47:06.336540 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.336513    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"]
Jul 29 19:47:06.461386 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.461353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istiod-ca-cert\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.461386 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.461394    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pczbh\" (UniqueName: \"kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-kube-api-access-pczbh\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.461664 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.461421    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-certs\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.461664 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.461508    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-credential-socket\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.461664 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.461545    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-data\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.461664 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.461601    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-socket\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.461833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.461669    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-token\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.461833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.461706    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-envoy\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.461833 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.461753    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-podinfo\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.562914 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.562868    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istiod-ca-cert\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563114 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.562921    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pczbh\" (UniqueName: \"kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-kube-api-access-pczbh\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563114 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.562950    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-certs\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563114 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.562975    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-credential-socket\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563114 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.562996    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-data\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563114 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.563030    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-socket\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563114 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.563082    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-token\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563435 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.563122    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-envoy\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563435 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.563181    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-podinfo\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.563435    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-credential-socket\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.563441    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-socket\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.563537    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-certs\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563710 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.563635    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-data\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.563710 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.563670    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istiod-ca-cert\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.565539 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.565516    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-envoy\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.565719 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.565701    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-podinfo\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.574843 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.574783    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pczbh\" (UniqueName: \"kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-kube-api-access-pczbh\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.576001 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.575978    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-token\") pod \"e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") " pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.638746 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.638716    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:06.777803 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.777779    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"]
Jul 29 19:47:06.779307 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:47:06.779284    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13fd67a_0902_40c4_8bc6_9e58ba09f54d.slice/crio-27f3978744d21ca13a7c0224ed5bd96b74d4ff73bd4f4e1d589ebbc345074b95 WatchSource:0}: Error finding container 27f3978744d21ca13a7c0224ed5bd96b74d4ff73bd4f4e1d589ebbc345074b95: Status 404 returned error can't find the container with id 27f3978744d21ca13a7c0224ed5bd96b74d4ff73bd4f4e1d589ebbc345074b95
Jul 29 19:47:06.781341 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.781311    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:47:06.781438 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.781382    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:47:06.781438 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:06.781429    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:47:07.738107 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:07.738069    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" event={"ID":"f13fd67a-0902-40c4-8bc6-9e58ba09f54d","Type":"ContainerStarted","Data":"7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d"}
Jul 29 19:47:07.738638 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:07.738120    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" event={"ID":"f13fd67a-0902-40c4-8bc6-9e58ba09f54d","Type":"ContainerStarted","Data":"27f3978744d21ca13a7c0224ed5bd96b74d4ff73bd4f4e1d589ebbc345074b95"}
Jul 29 19:47:07.762670 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:07.762616    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" podStartSLOduration=1.76259996 podStartE2EDuration="1.76259996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:47:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:47:07.760945403 +0000 UTC m=+1793.256706768" watchObservedRunningTime="2026-07-29 19:47:07.76259996 +0000 UTC m=+1793.258361301"
Jul 29 19:47:08.639205 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:08.639161    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:08.644882 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:08.644854    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:08.744488 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:08.744437    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:08.745598 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:08.745565    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:09.974811 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:09.974768    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"]
Jul 29 19:47:09.975363 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:09.975336    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy"
Jul 29 19:47:09.975363 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:09.975362    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d459b53-0473-4ec0-bf6b-c95cb1e146a7" containerName="istio-proxy"
Jul 29 19:47:09.978601 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:09.978582    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:09.981100 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:09.981074    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-ac4a75bf\""
Jul 29 19:47:09.981217 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:09.981152    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-ac4a75bf-dockercfg-k8294\""
Jul 29 19:47:09.990225 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:09.990199    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"]
Jul 29 19:47:10.103378 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.103335    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hr8v\" (UniqueName: \"kubernetes.io/projected/7b2e66cb-af5d-4eca-8c05-58985fc02008-kube-api-access-8hr8v\") pod \"payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr\" (UID: \"7b2e66cb-af5d-4eca-8c05-58985fc02008\") " pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:10.103605 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.103525    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b2e66cb-af5d-4eca-8c05-58985fc02008-plugins-config-volume\") pod \"payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr\" (UID: \"7b2e66cb-af5d-4eca-8c05-58985fc02008\") " pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:10.204997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.204957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b2e66cb-af5d-4eca-8c05-58985fc02008-plugins-config-volume\") pod \"payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr\" (UID: \"7b2e66cb-af5d-4eca-8c05-58985fc02008\") " pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:10.205186 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.205025    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hr8v\" (UniqueName: \"kubernetes.io/projected/7b2e66cb-af5d-4eca-8c05-58985fc02008-kube-api-access-8hr8v\") pod \"payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr\" (UID: \"7b2e66cb-af5d-4eca-8c05-58985fc02008\") " pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:10.205743 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.205721    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b2e66cb-af5d-4eca-8c05-58985fc02008-plugins-config-volume\") pod \"payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr\" (UID: \"7b2e66cb-af5d-4eca-8c05-58985fc02008\") " pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:10.213481 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.213442    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hr8v\" (UniqueName: \"kubernetes.io/projected/7b2e66cb-af5d-4eca-8c05-58985fc02008-kube-api-access-8hr8v\") pod \"payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr\" (UID: \"7b2e66cb-af5d-4eca-8c05-58985fc02008\") " pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:10.290790 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.290706    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:10.432950 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.432921    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"]
Jul 29 19:47:10.434977 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:47:10.434946    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b2e66cb_af5d_4eca_8c05_58985fc02008.slice/crio-f76cf0d96693e2f0b2001f836cfe1926a7578f6268d23d43be86ae98347a91e9 WatchSource:0}: Error finding container f76cf0d96693e2f0b2001f836cfe1926a7578f6268d23d43be86ae98347a91e9: Status 404 returned error can't find the container with id f76cf0d96693e2f0b2001f836cfe1926a7578f6268d23d43be86ae98347a91e9
Jul 29 19:47:10.759038 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.758998    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr" event={"ID":"7b2e66cb-af5d-4eca-8c05-58985fc02008","Type":"ContainerStarted","Data":"aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6"}
Jul 29 19:47:10.759226 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.759059    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr" event={"ID":"7b2e66cb-af5d-4eca-8c05-58985fc02008","Type":"ContainerStarted","Data":"f76cf0d96693e2f0b2001f836cfe1926a7578f6268d23d43be86ae98347a91e9"}
Jul 29 19:47:10.759226 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.759216    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:10.779956 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:10.779878    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr" podStartSLOduration=1.779863255 podStartE2EDuration="1.779863255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:47:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:47:10.779064832 +0000 UTC m=+1796.274826177" watchObservedRunningTime="2026-07-29 19:47:10.779863255 +0000 UTC m=+1796.275624644"
Jul 29 19:47:21.767866 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:21.767821    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:22.820920 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:22.820884    2566 generic.go:356] "Generic (PLEG): container finished" podID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerID="7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d" exitCode=0
Jul 29 19:47:22.821364 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:22.820962    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" event={"ID":"f13fd67a-0902-40c4-8bc6-9e58ba09f54d","Type":"ContainerDied","Data":"7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d"}
Jul 29 19:47:22.821364 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:22.821353    2566 scope.go:117] "RemoveContainer" containerID="7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d"
Jul 29 19:47:22.822355 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:22.822330    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:47:22.822418 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:22.822393    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:47:22.822476 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:22.822422    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:47:23.744411 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:23.744371    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:23.827973 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:23.827942    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" event={"ID":"f13fd67a-0902-40c4-8bc6-9e58ba09f54d","Type":"ContainerStarted","Data":"08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b"}
Jul 29 19:47:24.639902 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:24.639860    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:24.641589 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:24.641332    2566 patch_prober.go:28] interesting pod/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 19:47:24.641589 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:24.641518    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 19:47:25.639749 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:25.639702    2566 patch_prober.go:28] interesting pod/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 19:47:25.640261 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:25.639771    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 19:47:26.639957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:26.639908    2566 patch_prober.go:28] interesting pod/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 19:47:26.640435 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:26.639982    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 19:47:27.639725 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:27.639685    2566 patch_prober.go:28] interesting pod/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 29 19:47:27.640042 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:27.640008    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 29 19:47:28.643847 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:28.643818    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:28.644257 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:28.644097    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:28.644863 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:28.644843    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:39.964098 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:39.964058    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"]
Jul 29 19:47:39.964718 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:39.964317    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr" podUID="7b2e66cb-af5d-4eca-8c05-58985fc02008" containerName="payload-processing" containerID="cri-o://aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6" gracePeriod=30
Jul 29 19:47:40.227032 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.227001    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"]
Jul 29 19:47:40.227259 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.227207    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy" containerID="cri-o://08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b" gracePeriod=30
Jul 29 19:47:40.235050 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.235025    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:40.326597 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.326561    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hr8v\" (UniqueName: \"kubernetes.io/projected/7b2e66cb-af5d-4eca-8c05-58985fc02008-kube-api-access-8hr8v\") pod \"7b2e66cb-af5d-4eca-8c05-58985fc02008\" (UID: \"7b2e66cb-af5d-4eca-8c05-58985fc02008\") "
Jul 29 19:47:40.326790 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.326654    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b2e66cb-af5d-4eca-8c05-58985fc02008-plugins-config-volume\") pod \"7b2e66cb-af5d-4eca-8c05-58985fc02008\" (UID: \"7b2e66cb-af5d-4eca-8c05-58985fc02008\") "
Jul 29 19:47:40.327046 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.327012    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b2e66cb-af5d-4eca-8c05-58985fc02008-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7b2e66cb-af5d-4eca-8c05-58985fc02008" (UID: "7b2e66cb-af5d-4eca-8c05-58985fc02008"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:47:40.328669 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.328635    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b2e66cb-af5d-4eca-8c05-58985fc02008-kube-api-access-8hr8v" (OuterVolumeSpecName: "kube-api-access-8hr8v") pod "7b2e66cb-af5d-4eca-8c05-58985fc02008" (UID: "7b2e66cb-af5d-4eca-8c05-58985fc02008"). InnerVolumeSpecName "kube-api-access-8hr8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:47:40.428344 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.428307    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hr8v\" (UniqueName: \"kubernetes.io/projected/7b2e66cb-af5d-4eca-8c05-58985fc02008-kube-api-access-8hr8v\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:40.428344 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.428342    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b2e66cb-af5d-4eca-8c05-58985fc02008-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:40.917428 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.917393    2566 generic.go:356] "Generic (PLEG): container finished" podID="7b2e66cb-af5d-4eca-8c05-58985fc02008" containerID="aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6" exitCode=0
Jul 29 19:47:40.917649 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.917445    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"
Jul 29 19:47:40.917649 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.917481    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr" event={"ID":"7b2e66cb-af5d-4eca-8c05-58985fc02008","Type":"ContainerDied","Data":"aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6"}
Jul 29 19:47:40.917649 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.917536    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr" event={"ID":"7b2e66cb-af5d-4eca-8c05-58985fc02008","Type":"ContainerDied","Data":"f76cf0d96693e2f0b2001f836cfe1926a7578f6268d23d43be86ae98347a91e9"}
Jul 29 19:47:40.917649 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.917558    2566 scope.go:117] "RemoveContainer" containerID="aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6"
Jul 29 19:47:40.926656 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.926638    2566 scope.go:117] "RemoveContainer" containerID="aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6"
Jul 29 19:47:40.927014 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:47:40.926993    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6\": container with ID starting with aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6 not found: ID does not exist" containerID="aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6"
Jul 29 19:47:40.927109 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.927025    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6"} err="failed to get container status \"aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6\": rpc error: code = NotFound desc = could not find container \"aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6\": container with ID starting with aef6c276e3be048465d779b372e8889ae494c0a2ccee18429cdac573d1998ca6 not found: ID does not exist"
Jul 29 19:47:40.940362 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.940333    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"]
Jul 29 19:47:40.953898 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:40.953866    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ac4a75bf-6dcf95dd86-7k5pr"]
Jul 29 19:47:41.101886 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:41.101851    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b2e66cb-af5d-4eca-8c05-58985fc02008" path="/var/lib/kubelet/pods/7b2e66cb-af5d-4eca-8c05-58985fc02008/volumes"
Jul 29 19:47:43.646730 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:43.646688    2566 patch_prober.go:28] interesting pod/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 19:47:43.647217 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:43.646760    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 19:47:45.490170 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.490114    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:45.581146 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581113    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-certs\") pod \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") "
Jul 29 19:47:45.581146 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581159    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-token\") pod \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") "
Jul 29 19:47:45.581404 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581249    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istiod-ca-cert\") pod \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") "
Jul 29 19:47:45.581404 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581273    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pczbh\" (UniqueName: \"kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-kube-api-access-pczbh\") pod \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") "
Jul 29 19:47:45.581404 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581322    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-credential-socket\") pod \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") "
Jul 29 19:47:45.581593 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581408    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f13fd67a-0902-40c4-8bc6-9e58ba09f54d" (UID: "f13fd67a-0902-40c4-8bc6-9e58ba09f54d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:47:45.581593 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581441    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-envoy\") pod \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") "
Jul 29 19:47:45.581593 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581514    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-podinfo\") pod \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") "
Jul 29 19:47:45.581593 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581545    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-socket\") pod \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") "
Jul 29 19:47:45.581593 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581575    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-data\") pod \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\" (UID: \"f13fd67a-0902-40c4-8bc6-9e58ba09f54d\") "
Jul 29 19:47:45.581860 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581648    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f13fd67a-0902-40c4-8bc6-9e58ba09f54d" (UID: "f13fd67a-0902-40c4-8bc6-9e58ba09f54d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:47:45.581860 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581715    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f13fd67a-0902-40c4-8bc6-9e58ba09f54d" (UID: "f13fd67a-0902-40c4-8bc6-9e58ba09f54d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:47:45.581971 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.581865    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f13fd67a-0902-40c4-8bc6-9e58ba09f54d" (UID: "f13fd67a-0902-40c4-8bc6-9e58ba09f54d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:47:45.582107 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.582086    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-credential-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:45.582171 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.582115    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:45.582171 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.582132    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-workload-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:45.582171 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.582146    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istiod-ca-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:45.582621 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.582598    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-data" (OuterVolumeSpecName: "istio-data") pod "f13fd67a-0902-40c4-8bc6-9e58ba09f54d" (UID: "f13fd67a-0902-40c4-8bc6-9e58ba09f54d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:47:45.583660 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.583628    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f13fd67a-0902-40c4-8bc6-9e58ba09f54d" (UID: "f13fd67a-0902-40c4-8bc6-9e58ba09f54d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:47:45.583773 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.583747    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f13fd67a-0902-40c4-8bc6-9e58ba09f54d" (UID: "f13fd67a-0902-40c4-8bc6-9e58ba09f54d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 19:47:45.583937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.583913    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-token" (OuterVolumeSpecName: "istio-token") pod "f13fd67a-0902-40c4-8bc6-9e58ba09f54d" (UID: "f13fd67a-0902-40c4-8bc6-9e58ba09f54d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:47:45.584011 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.583939    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-kube-api-access-pczbh" (OuterVolumeSpecName: "kube-api-access-pczbh") pod "f13fd67a-0902-40c4-8bc6-9e58ba09f54d" (UID: "f13fd67a-0902-40c4-8bc6-9e58ba09f54d"). InnerVolumeSpecName "kube-api-access-pczbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:47:45.682957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.682927    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-envoy\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:45.682957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.682954    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-podinfo\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:45.682957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.682964    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-data\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:45.683190 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.682972    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-istio-token\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:45.683190 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.682980    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pczbh\" (UniqueName: \"kubernetes.io/projected/f13fd67a-0902-40c4-8bc6-9e58ba09f54d-kube-api-access-pczbh\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:47:45.946909 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.946872    2566 generic.go:356] "Generic (PLEG): container finished" podID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerID="08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b" exitCode=0
Jul 29 19:47:45.947097 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.946925    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"
Jul 29 19:47:45.947097 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.946952    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" event={"ID":"f13fd67a-0902-40c4-8bc6-9e58ba09f54d","Type":"ContainerDied","Data":"08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b"}
Jul 29 19:47:45.947097 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.947013    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq" event={"ID":"f13fd67a-0902-40c4-8bc6-9e58ba09f54d","Type":"ContainerDied","Data":"27f3978744d21ca13a7c0224ed5bd96b74d4ff73bd4f4e1d589ebbc345074b95"}
Jul 29 19:47:45.947097 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.947043    2566 scope.go:117] "RemoveContainer" containerID="08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b"
Jul 29 19:47:45.957073 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.957048    2566 scope.go:117] "RemoveContainer" containerID="7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d"
Jul 29 19:47:45.964896 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.964878    2566 scope.go:117] "RemoveContainer" containerID="08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b"
Jul 29 19:47:45.965166 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:47:45.965147    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b\": container with ID starting with 08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b not found: ID does not exist" containerID="08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b"
Jul 29 19:47:45.965225 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.965174    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b"} err="failed to get container status \"08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b\": rpc error: code = NotFound desc = could not find container \"08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b\": container with ID starting with 08663d49f85d46b8b7567d83c2e8d04ae2eb875e167edc2b07f6b1280f36fc7b not found: ID does not exist"
Jul 29 19:47:45.965225 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.965192    2566 scope.go:117] "RemoveContainer" containerID="7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d"
Jul 29 19:47:45.965422 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:47:45.965407    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d\": container with ID starting with 7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d not found: ID does not exist" containerID="7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d"
Jul 29 19:47:45.965500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.965425    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d"} err="failed to get container status \"7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d\": rpc error: code = NotFound desc = could not find container \"7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d\": container with ID starting with 7761807ca664f79e092a197353bb1c308d781e8278febffcba4541eb635ee74d not found: ID does not exist"
Jul 29 19:47:45.970849 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.970827    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"]
Jul 29 19:47:45.975000 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:45.974976    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ac4a75bf-openshift-default-6bc76bcdcb-h44tq"]
Jul 29 19:47:47.103478 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:47:47.103427    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" path="/var/lib/kubelet/pods/f13fd67a-0902-40c4-8bc6-9e58ba09f54d/volumes"
Jul 29 19:49:38.386518 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.386479    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"]
Jul 29 19:49:38.386997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.386859    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy"
Jul 29 19:49:38.386997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.386870    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy"
Jul 29 19:49:38.386997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.386886    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b2e66cb-af5d-4eca-8c05-58985fc02008" containerName="payload-processing"
Jul 29 19:49:38.386997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.386892    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b2e66cb-af5d-4eca-8c05-58985fc02008" containerName="payload-processing"
Jul 29 19:49:38.386997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.386906    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy"
Jul 29 19:49:38.386997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.386911    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy"
Jul 29 19:49:38.386997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.386971    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b2e66cb-af5d-4eca-8c05-58985fc02008" containerName="payload-processing"
Jul 29 19:49:38.386997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.386980    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy"
Jul 29 19:49:38.386997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.386986    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13fd67a-0902-40c4-8bc6-9e58ba09f54d" containerName="istio-proxy"
Jul 29 19:49:38.390163 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.390147    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.392854 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.392833    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 19:49:38.392961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.392832    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cd17a221-openshift-default-dockercfg-7fnwz\""
Jul 29 19:49:38.415739 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.415713    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"]
Jul 29 19:49:38.550047 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.550010    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-envoy\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.550047 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.550048    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-podinfo\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.550263 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.550068    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9szhp\" (UniqueName: \"kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-kube-api-access-9szhp\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.550263 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.550094    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-socket\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.550263 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.550169    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-credential-socket\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.550263 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.550217    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-certs\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.550522 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.550281    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-token\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.550522 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.550334    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istiod-ca-cert\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.550522 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.550369    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-data\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.651490 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.651382    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-token\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.651490 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.651477    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istiod-ca-cert\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.651704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.651522    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-data\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.651704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.651557    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-envoy\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.651704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.651584    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-podinfo\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.651704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.651613    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9szhp\" (UniqueName: \"kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-kube-api-access-9szhp\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.651704 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.651638    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-socket\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.651951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.651737    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-credential-socket\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.651951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.651811    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-certs\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.652174 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.652148    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-data\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.652309 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.652193    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-credential-socket\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.652309 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.652192    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-certs\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.652429 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.652309    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istiod-ca-cert\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.652429 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.652362    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-socket\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.654001 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.653977    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-envoy\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.654177 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.654159    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-podinfo\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.661898 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.661874    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-token\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.662057 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.661918    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9szhp\" (UniqueName: \"kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-kube-api-access-9szhp\") pod \"e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") " pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.699792 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.699759    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:38.846660 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.846633    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"]
Jul 29 19:49:38.849374 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:49:38.849347    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91966cf4_cc3f_410d_b6b6_ca253c8d3a9a.slice/crio-2932caad79ccc801ee4d6309f072fa9a909322bfb46648888ff83243a5414046 WatchSource:0}: Error finding container 2932caad79ccc801ee4d6309f072fa9a909322bfb46648888ff83243a5414046: Status 404 returned error can't find the container with id 2932caad79ccc801ee4d6309f072fa9a909322bfb46648888ff83243a5414046
Jul 29 19:49:38.854998 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.852930    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:49:38.854998 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.853019    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:49:38.854998 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:38.853060    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:49:39.495464 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:39.495413    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" event={"ID":"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a","Type":"ContainerStarted","Data":"4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533"}
Jul 29 19:49:39.495956 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:39.495493    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" event={"ID":"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a","Type":"ContainerStarted","Data":"2932caad79ccc801ee4d6309f072fa9a909322bfb46648888ff83243a5414046"}
Jul 29 19:49:39.574929 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:39.574873    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" podStartSLOduration=1.574855628 podStartE2EDuration="1.574855628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:49:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:49:39.573643542 +0000 UTC m=+1945.069404885" watchObservedRunningTime="2026-07-29 19:49:39.574855628 +0000 UTC m=+1945.070617048"
Jul 29 19:49:39.700459 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:39.700416    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:39.708265 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:39.708239    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:40.500247 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:40.500215    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:40.501100 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:40.501083    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:52.561587 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:52.561553    2566 generic.go:356] "Generic (PLEG): container finished" podID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerID="4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533" exitCode=0
Jul 29 19:49:52.561989 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:52.561621    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" event={"ID":"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a","Type":"ContainerDied","Data":"4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533"}
Jul 29 19:49:52.562036 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:52.561987    2566 scope.go:117] "RemoveContainer" containerID="4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533"
Jul 29 19:49:52.563062 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:52.563034    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:49:52.563152 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:52.563121    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:49:52.563216 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:52.563154    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:49:53.569409 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:53.569377    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" event={"ID":"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a","Type":"ContainerStarted","Data":"f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792"}
Jul 29 19:49:53.700280 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:53.700241    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:53.701472 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:53.701421    2566 patch_prober.go:28] interesting pod/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 19:49:53.701605 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:53.701499    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 19:49:54.700753 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:54.700716    2566 patch_prober.go:28] interesting pod/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 19:49:54.701130 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:54.700779    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 19:49:55.500726 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:55.500696    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:55.700947 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:55.700908    2566 patch_prober.go:28] interesting pod/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 19:49:55.701311 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:55.700999    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 19:49:56.700247 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:56.700211    2566 patch_prober.go:28] interesting pod/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 29 19:49:56.700415 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:56.700280    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 29 19:49:57.703656 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:57.703621    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:49:57.704461 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:49:57.704434    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:50:31.066242 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:31.066159    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"]
Jul 29 19:50:31.066794 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:31.066376    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerName="istio-proxy" containerID="cri-o://f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792" gracePeriod=30
Jul 29 19:50:36.319827 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.319797    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:50:36.397664 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.397585    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-podinfo\") pod \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") "
Jul 29 19:50:36.397664 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.397627    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9szhp\" (UniqueName: \"kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-kube-api-access-9szhp\") pod \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") "
Jul 29 19:50:36.397664 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.397656    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-certs\") pod \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") "
Jul 29 19:50:36.397921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.397676    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-data\") pod \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") "
Jul 29 19:50:36.397921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.397704    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-credential-socket\") pod \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") "
Jul 29 19:50:36.397921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.397768    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-envoy\") pod \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") "
Jul 29 19:50:36.397921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.397797    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-token\") pod \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") "
Jul 29 19:50:36.397921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.397887    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istiod-ca-cert\") pod \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") "
Jul 29 19:50:36.397921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.397911    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-socket\") pod \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\" (UID: \"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a\") "
Jul 29 19:50:36.398227 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.397976    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" (UID: "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:50:36.398227 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.398066    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" (UID: "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:50:36.398227 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.398223    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:50:36.398371 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.398240    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-credential-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:50:36.398371 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.398275    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" (UID: "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:50:36.398371 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.398303    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" (UID: "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:50:36.398825 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.398795    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-data" (OuterVolumeSpecName: "istio-data") pod "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" (UID: "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:50:36.400011 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.399990    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" (UID: "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 19:50:36.400092 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.399999    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-token" (OuterVolumeSpecName: "istio-token") pod "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" (UID: "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:50:36.400287 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.400272    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-kube-api-access-9szhp" (OuterVolumeSpecName: "kube-api-access-9szhp") pod "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" (UID: "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a"). InnerVolumeSpecName "kube-api-access-9szhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:50:36.400339 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.400280    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" (UID: "91966cf4-cc3f-410d-b6b6-ca253c8d3a9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:50:36.499485 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.499427    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istiod-ca-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:50:36.499485 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.499483    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-workload-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:50:36.499646 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.499494    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-podinfo\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:50:36.499646 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.499503    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9szhp\" (UniqueName: \"kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-kube-api-access-9szhp\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:50:36.499646 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.499513    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-data\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:50:36.499646 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.499521    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-envoy\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:50:36.499646 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.499528    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a-istio-token\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:50:36.789887 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.789851    2566 generic.go:356] "Generic (PLEG): container finished" podID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerID="f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792" exitCode=0
Jul 29 19:50:36.790051 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.789904    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"
Jul 29 19:50:36.790051 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.789936    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" event={"ID":"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a","Type":"ContainerDied","Data":"f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792"}
Jul 29 19:50:36.790051 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.789998    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2" event={"ID":"91966cf4-cc3f-410d-b6b6-ca253c8d3a9a","Type":"ContainerDied","Data":"2932caad79ccc801ee4d6309f072fa9a909322bfb46648888ff83243a5414046"}
Jul 29 19:50:36.790051 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.790022    2566 scope.go:117] "RemoveContainer" containerID="f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792"
Jul 29 19:50:36.799523 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.799500    2566 scope.go:117] "RemoveContainer" containerID="4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533"
Jul 29 19:50:36.807317 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.807300    2566 scope.go:117] "RemoveContainer" containerID="f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792"
Jul 29 19:50:36.807600 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:50:36.807580    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792\": container with ID starting with f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792 not found: ID does not exist" containerID="f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792"
Jul 29 19:50:36.807647 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.807609    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792"} err="failed to get container status \"f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792\": rpc error: code = NotFound desc = could not find container \"f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792\": container with ID starting with f1dd6912638fdde9022de82295d04dbe5e742f1b2730522afedc035cb3aa6792 not found: ID does not exist"
Jul 29 19:50:36.807647 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.807627    2566 scope.go:117] "RemoveContainer" containerID="4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533"
Jul 29 19:50:36.807879 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:50:36.807862    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533\": container with ID starting with 4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533 not found: ID does not exist" containerID="4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533"
Jul 29 19:50:36.807917 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.807886    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533"} err="failed to get container status \"4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533\": rpc error: code = NotFound desc = could not find container \"4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533\": container with ID starting with 4308b057db06eb6905df1e83b1415086cb0a96388c22caffdb9914f5cf962533 not found: ID does not exist"
Jul 29 19:50:36.814158 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.814138    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"]
Jul 29 19:50:36.817513 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:36.817493    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cd17a221-openshift-default-588ff7c78d-qdfh2"]
Jul 29 19:50:37.101800 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:37.101719    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" path="/var/lib/kubelet/pods/91966cf4-cc3f-410d-b6b6-ca253c8d3a9a/volumes"
Jul 29 19:50:51.503364 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.503330    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"]
Jul 29 19:50:51.503759 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.503736    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerName="istio-proxy"
Jul 29 19:50:51.503759 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.503748    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerName="istio-proxy"
Jul 29 19:50:51.503845 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.503833    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerName="istio-proxy"
Jul 29 19:50:51.503845 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.503842    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerName="istio-proxy"
Jul 29 19:50:51.506850 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.506834    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.509646 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.509626    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6757cc40-openshift-default-dockercfg-jkssw\""
Jul 29 19:50:51.509779 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.509640    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 19:50:51.526694 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.526673    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"]
Jul 29 19:50:51.632157 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.632127    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b752b8e7-893d-4840-9516-4759aa870947-istiod-ca-cert\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.632325 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.632177    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-certs\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.632325 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.632237    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-socket\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.632325 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.632262    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-data\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.632325 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.632296    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-istio-token\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.632325 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.632321    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b752b8e7-893d-4840-9516-4759aa870947-istio-podinfo\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.632519 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.632376    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-credential-socket\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.632519 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.632403    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-envoy\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.632519 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.632439    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbgnd\" (UniqueName: \"kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-kube-api-access-gbgnd\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.733683 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.733650    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-certs\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.733860 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.733702    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-socket\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.733860 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.733721    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-data\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.733860 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.733748    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-istio-token\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.733860 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.733782    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b752b8e7-893d-4840-9516-4759aa870947-istio-podinfo\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.733860 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.733809    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-credential-socket\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.733860 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.733835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-envoy\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.734133 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.733863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbgnd\" (UniqueName: \"kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-kube-api-access-gbgnd\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.734133 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.733905    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b752b8e7-893d-4840-9516-4759aa870947-istiod-ca-cert\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.734133 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.734092    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-socket\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.734336 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.734315    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-certs\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.734400 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.734384    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-credential-socket\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.734478 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.734403    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-data\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.734585 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.734561    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b752b8e7-893d-4840-9516-4759aa870947-istiod-ca-cert\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.735978 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.735956    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-envoy\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.736268 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.736250    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b752b8e7-893d-4840-9516-4759aa870947-istio-podinfo\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.744224 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.744201    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-istio-token\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.744832 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.744800    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbgnd\" (UniqueName: \"kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-kube-api-access-gbgnd\") pod \"e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") " pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.816845 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.816775    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:51.957634 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.957605    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"]
Jul 29 19:50:51.959106 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:50:51.959079    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb752b8e7_893d_4840_9516_4759aa870947.slice/crio-eb33a54a45f897ab438e434563e4f7dbd70c77039f28f932e5896eef1dc9a4bb WatchSource:0}: Error finding container eb33a54a45f897ab438e434563e4f7dbd70c77039f28f932e5896eef1dc9a4bb: Status 404 returned error can't find the container with id eb33a54a45f897ab438e434563e4f7dbd70c77039f28f932e5896eef1dc9a4bb
Jul 29 19:50:51.961221 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.961186    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:50:51.961294 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.961255    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:50:51.961294 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:51.961283    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:50:52.870391 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:52.870350    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n" event={"ID":"b752b8e7-893d-4840-9516-4759aa870947","Type":"ContainerStarted","Data":"ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f"}
Jul 29 19:50:52.870801 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:52.870468    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n" event={"ID":"b752b8e7-893d-4840-9516-4759aa870947","Type":"ContainerStarted","Data":"eb33a54a45f897ab438e434563e4f7dbd70c77039f28f932e5896eef1dc9a4bb"}
Jul 29 19:50:52.898251 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:52.898208    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n" podStartSLOduration=1.898195842 podStartE2EDuration="1.898195842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:50:52.898045555 +0000 UTC m=+2018.393806923" watchObservedRunningTime="2026-07-29 19:50:52.898195842 +0000 UTC m=+2018.393957182"
Jul 29 19:50:53.817932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:53.817896    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:53.822873 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:53.822849    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:53.875370 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:53.875340    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:50:53.876139 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:50:53.876117    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:51:07.945653 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:07.945616    2566 generic.go:356] "Generic (PLEG): container finished" podID="b752b8e7-893d-4840-9516-4759aa870947" containerID="ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f" exitCode=0
Jul 29 19:51:07.946098 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:07.945695    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n" event={"ID":"b752b8e7-893d-4840-9516-4759aa870947","Type":"ContainerDied","Data":"ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f"}
Jul 29 19:51:07.946141 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:07.946096    2566 scope.go:117] "RemoveContainer" containerID="ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f"
Jul 29 19:51:07.947099 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:07.947072    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:51:07.947224 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:07.947153    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:51:07.947224 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:07.947186    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:51:08.875583 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:08.875540    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:51:08.955709 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:08.955665    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n" event={"ID":"b752b8e7-893d-4840-9516-4759aa870947","Type":"ContainerStarted","Data":"b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b"}
Jul 29 19:51:09.817346 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:09.817310    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:51:09.818468 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:09.818428    2566 patch_prober.go:28] interesting pod/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 29 19:51:09.818571 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:09.818500    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n" podUID="b752b8e7-893d-4840-9516-4759aa870947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 29 19:51:10.817505 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:10.817443    2566 patch_prober.go:28] interesting pod/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 29 19:51:10.817921 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:10.817533    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n" podUID="b752b8e7-893d-4840-9516-4759aa870947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 29 19:51:11.817392 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:11.817350    2566 patch_prober.go:28] interesting pod/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Jul 29 19:51:11.817580 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:11.817421    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n" podUID="b752b8e7-893d-4840-9516-4759aa870947" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Jul 29 19:51:12.820987 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:12.820955    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:51:12.821370 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:12.821223    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:51:12.821919 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:12.821904    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:51:38.265732 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.265647    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"]
Jul 29 19:51:38.266321 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.266223    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerName="istio-proxy"
Jul 29 19:51:38.266321 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.266244    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="91966cf4-cc3f-410d-b6b6-ca253c8d3a9a" containerName="istio-proxy"
Jul 29 19:51:38.269396 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.269376    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.272199 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.272178    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6757cc40\"/\"openshift-service-ca.crt\""
Jul 29 19:51:38.272305 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.272177    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6757cc40\"/\"kube-root-ca.crt\""
Jul 29 19:51:38.273355 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.273336    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6757cc40\"/\"default-dockercfg-5m7dq\""
Jul 29 19:51:38.273422 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.273346    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6757cc40\"/\"e2e-lifecycle-model-6757cc40-kserve-self-signed-certs\""
Jul 29 19:51:38.279932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.279915    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"]
Jul 29 19:51:38.312116 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.312089    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-dshm\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.312257 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.312134    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjjmn\" (UniqueName: \"kubernetes.io/projected/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-kube-api-access-zjjmn\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.312257 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.312161    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-model-cache\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.312257 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.312240    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tls-certs\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.312385 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.312314    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tmp-dir\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.312385 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.312351    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-home\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.413909 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.413698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-dshm\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.413909 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.413769    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjjmn\" (UniqueName: \"kubernetes.io/projected/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-kube-api-access-zjjmn\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.413909 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.413801    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-model-cache\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.413909 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.413851    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tls-certs\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.414312 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.414089    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tmp-dir\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.414312 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.414205    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-home\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.414512 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.414489    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-model-cache\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.414600 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.414576    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tmp-dir\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.414680 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.414643    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-home\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.416160 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.416139    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-dshm\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.416439 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.416419    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tls-certs\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.422186 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.422167    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjjmn\" (UniqueName: \"kubernetes.io/projected/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-kube-api-access-zjjmn\") pod \"e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") " pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.579474 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.579378    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:38.713322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:38.713299    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"]
Jul 29 19:51:38.715240 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:51:38.715212    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb1292f5_2a7b_4290_85e8_d9f39c31b8b1.slice/crio-1dce78e32fe7e894076cf66c679cc223b9eb88d1dbe551f24823dfd9f9b03145 WatchSource:0}: Error finding container 1dce78e32fe7e894076cf66c679cc223b9eb88d1dbe551f24823dfd9f9b03145: Status 404 returned error can't find the container with id 1dce78e32fe7e894076cf66c679cc223b9eb88d1dbe551f24823dfd9f9b03145
Jul 29 19:51:39.105254 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:39.105224    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6" event={"ID":"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1","Type":"ContainerStarted","Data":"8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e"}
Jul 29 19:51:39.105254 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:39.105260    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6" event={"ID":"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1","Type":"ContainerStarted","Data":"1dce78e32fe7e894076cf66c679cc223b9eb88d1dbe551f24823dfd9f9b03145"}
Jul 29 19:51:39.126117 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:39.126069    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6" podStartSLOduration=1.126053587 podStartE2EDuration="1.126053587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:51:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:51:39.125071417 +0000 UTC m=+2064.620832760" watchObservedRunningTime="2026-07-29 19:51:39.126053587 +0000 UTC m=+2064.621814956"
Jul 29 19:51:48.580194 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:48.580162    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:48.580619 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:48.580213    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:48.592545 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:48.592523    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:51:49.161623 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:51:49.161589    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:52:22.863166 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:22.863128    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"]
Jul 29 19:52:37.882780 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:37.882731    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6" podUID="eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" containerName="main" containerID="cri-o://8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e" gracePeriod=45
Jul 29 19:52:38.132560 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.132534    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:52:38.209335 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209306    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tmp-dir\") pod \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") "
Jul 29 19:52:38.209335 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209336    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-home\") pod \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") "
Jul 29 19:52:38.209600 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209382    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjjmn\" (UniqueName: \"kubernetes.io/projected/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-kube-api-access-zjjmn\") pod \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") "
Jul 29 19:52:38.209600 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209441    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-model-cache\") pod \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") "
Jul 29 19:52:38.209600 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209513    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tls-certs\") pod \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") "
Jul 29 19:52:38.209600 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209549    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-dshm\") pod \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\" (UID: \"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1\") "
Jul 29 19:52:38.209822 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209677    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" (UID: "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:52:38.209822 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209710    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-home" (OuterVolumeSpecName: "home") pod "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" (UID: "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:52:38.209822 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209759    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-model-cache" (OuterVolumeSpecName: "model-cache") pod "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" (UID: "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:52:38.209981 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209934    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tmp-dir\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:52:38.209981 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209953    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-home\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:52:38.209981 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.209967    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-model-cache\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:52:38.211666 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.211639    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" (UID: "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:52:38.212021 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.212005    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-kube-api-access-zjjmn" (OuterVolumeSpecName: "kube-api-access-zjjmn") pod "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" (UID: "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1"). InnerVolumeSpecName "kube-api-access-zjjmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:52:38.212076 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.212022    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-dshm" (OuterVolumeSpecName: "dshm") pod "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" (UID: "eb1292f5-2a7b-4290-85e8-d9f39c31b8b1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:52:38.310703 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.310657    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-tls-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:52:38.310703 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.310700    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-dshm\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:52:38.310703 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.310710    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjjmn\" (UniqueName: \"kubernetes.io/projected/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1-kube-api-access-zjjmn\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:52:38.404117 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.404083    2566 generic.go:356] "Generic (PLEG): container finished" podID="eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" containerID="8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e" exitCode=0
Jul 29 19:52:38.404311 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.404148    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"
Jul 29 19:52:38.404311 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.404161    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6" event={"ID":"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1","Type":"ContainerDied","Data":"8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e"}
Jul 29 19:52:38.404311 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.404211    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6" event={"ID":"eb1292f5-2a7b-4290-85e8-d9f39c31b8b1","Type":"ContainerDied","Data":"1dce78e32fe7e894076cf66c679cc223b9eb88d1dbe551f24823dfd9f9b03145"}
Jul 29 19:52:38.404311 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.404231    2566 scope.go:117] "RemoveContainer" containerID="8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e"
Jul 29 19:52:38.413254 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.413233    2566 scope.go:117] "RemoveContainer" containerID="8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e"
Jul 29 19:52:38.413609 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:52:38.413560    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e\": container with ID starting with 8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e not found: ID does not exist" containerID="8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e"
Jul 29 19:52:38.413609 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.413590    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e"} err="failed to get container status \"8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e\": rpc error: code = NotFound desc = could not find container \"8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e\": container with ID starting with 8806a03d8baedce675cd11a1d9a49204d408a756de1f42bb4ece83968bdfd25e not found: ID does not exist"
Jul 29 19:52:38.427000 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.426969    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"]
Jul 29 19:52:38.430238 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:38.430212    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6757cc40/e2e-lifecycle-model-6757cc40-kserve-66dcf97cf9-9hhl6"]
Jul 29 19:52:39.102326 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:52:39.102285    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" path="/var/lib/kubelet/pods/eb1292f5-2a7b-4290-85e8-d9f39c31b8b1/volumes"
Jul 29 19:53:21.170009 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:21.169979    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"]
Jul 29 19:53:21.170593 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:21.170197    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n" podUID="b752b8e7-893d-4840-9516-4759aa870947" containerName="istio-proxy" containerID="cri-o://b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b" gracePeriod=30
Jul 29 19:53:26.426163 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.426137    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:53:26.467140 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467059    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-credential-socket\") pod \"b752b8e7-893d-4840-9516-4759aa870947\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") "
Jul 29 19:53:26.467140 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467120    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-certs\") pod \"b752b8e7-893d-4840-9516-4759aa870947\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") "
Jul 29 19:53:26.467328 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467166    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b752b8e7-893d-4840-9516-4759aa870947-istiod-ca-cert\") pod \"b752b8e7-893d-4840-9516-4759aa870947\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") "
Jul 29 19:53:26.467328 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467212    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b752b8e7-893d-4840-9516-4759aa870947-istio-podinfo\") pod \"b752b8e7-893d-4840-9516-4759aa870947\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") "
Jul 29 19:53:26.467431 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467352    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b752b8e7-893d-4840-9516-4759aa870947" (UID: "b752b8e7-893d-4840-9516-4759aa870947"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:53:26.467431 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467363    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-envoy\") pod \"b752b8e7-893d-4840-9516-4759aa870947\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") "
Jul 29 19:53:26.467651 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467420    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b752b8e7-893d-4840-9516-4759aa870947" (UID: "b752b8e7-893d-4840-9516-4759aa870947"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:53:26.467651 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467495    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbgnd\" (UniqueName: \"kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-kube-api-access-gbgnd\") pod \"b752b8e7-893d-4840-9516-4759aa870947\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") "
Jul 29 19:53:26.467651 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467530    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-data\") pod \"b752b8e7-893d-4840-9516-4759aa870947\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") "
Jul 29 19:53:26.467651 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467576    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-socket\") pod \"b752b8e7-893d-4840-9516-4759aa870947\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") "
Jul 29 19:53:26.467651 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467599    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-istio-token\") pod \"b752b8e7-893d-4840-9516-4759aa870947\" (UID: \"b752b8e7-893d-4840-9516-4759aa870947\") "
Jul 29 19:53:26.468006 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467651    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b752b8e7-893d-4840-9516-4759aa870947-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b752b8e7-893d-4840-9516-4759aa870947" (UID: "b752b8e7-893d-4840-9516-4759aa870947"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:53:26.468006 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467941    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b752b8e7-893d-4840-9516-4759aa870947" (UID: "b752b8e7-893d-4840-9516-4759aa870947"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:53:26.468006 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467961    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b752b8e7-893d-4840-9516-4759aa870947-istiod-ca-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:53:26.468006 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467980    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-credential-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:53:26.468006 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.467995    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:53:26.468768 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.468725    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-data" (OuterVolumeSpecName: "istio-data") pod "b752b8e7-893d-4840-9516-4759aa870947" (UID: "b752b8e7-893d-4840-9516-4759aa870947"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:53:26.470227 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.470204    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-kube-api-access-gbgnd" (OuterVolumeSpecName: "kube-api-access-gbgnd") pod "b752b8e7-893d-4840-9516-4759aa870947" (UID: "b752b8e7-893d-4840-9516-4759aa870947"). InnerVolumeSpecName "kube-api-access-gbgnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:53:26.470405 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.470383    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b752b8e7-893d-4840-9516-4759aa870947" (UID: "b752b8e7-893d-4840-9516-4759aa870947"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:53:26.470876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.470855    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b752b8e7-893d-4840-9516-4759aa870947-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b752b8e7-893d-4840-9516-4759aa870947" (UID: "b752b8e7-893d-4840-9516-4759aa870947"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 19:53:26.471133 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.471107    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-istio-token" (OuterVolumeSpecName: "istio-token") pod "b752b8e7-893d-4840-9516-4759aa870947" (UID: "b752b8e7-893d-4840-9516-4759aa870947"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:53:26.568430 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.568401    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b752b8e7-893d-4840-9516-4759aa870947-istio-podinfo\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:53:26.568430 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.568426    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-envoy\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:53:26.568430 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.568435    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbgnd\" (UniqueName: \"kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-kube-api-access-gbgnd\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:53:26.568640 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.568445    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-istio-data\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:53:26.568640 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.568466    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b752b8e7-893d-4840-9516-4759aa870947-workload-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:53:26.568640 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.568474    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b752b8e7-893d-4840-9516-4759aa870947-istio-token\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:53:26.631462 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.631412    2566 generic.go:356] "Generic (PLEG): container finished" podID="b752b8e7-893d-4840-9516-4759aa870947" containerID="b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b" exitCode=0
Jul 29 19:53:26.631614 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.631480    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"
Jul 29 19:53:26.631614 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.631501    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n" event={"ID":"b752b8e7-893d-4840-9516-4759aa870947","Type":"ContainerDied","Data":"b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b"}
Jul 29 19:53:26.631614 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.631584    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n" event={"ID":"b752b8e7-893d-4840-9516-4759aa870947","Type":"ContainerDied","Data":"eb33a54a45f897ab438e434563e4f7dbd70c77039f28f932e5896eef1dc9a4bb"}
Jul 29 19:53:26.631614 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.631607    2566 scope.go:117] "RemoveContainer" containerID="b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b"
Jul 29 19:53:26.641271 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.641253    2566 scope.go:117] "RemoveContainer" containerID="ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f"
Jul 29 19:53:26.649112 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.649096    2566 scope.go:117] "RemoveContainer" containerID="b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b"
Jul 29 19:53:26.649372 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:53:26.649350    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b\": container with ID starting with b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b not found: ID does not exist" containerID="b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b"
Jul 29 19:53:26.649439 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.649379    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b"} err="failed to get container status \"b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b\": rpc error: code = NotFound desc = could not find container \"b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b\": container with ID starting with b2f4c79851097e89437bdbaf8d0dd0e340c2434c8507148fdd5e373989c4e02b not found: ID does not exist"
Jul 29 19:53:26.649439 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.649396    2566 scope.go:117] "RemoveContainer" containerID="ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f"
Jul 29 19:53:26.649662 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:53:26.649643    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f\": container with ID starting with ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f not found: ID does not exist" containerID="ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f"
Jul 29 19:53:26.649707 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.649668    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f"} err="failed to get container status \"ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f\": rpc error: code = NotFound desc = could not find container \"ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f\": container with ID starting with ba8613ed73a5dfd1f88560c7dee280185408d4005036769eec94525fcc54a05f not found: ID does not exist"
Jul 29 19:53:26.656751 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.656728    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"]
Jul 29 19:53:26.660622 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:26.660601    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6757cc40-openshift-default-5795d7b75b-2wb6n"]
Jul 29 19:53:27.102955 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:53:27.102918    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b752b8e7-893d-4840-9516-4759aa870947" path="/var/lib/kubelet/pods/b752b8e7-893d-4840-9516-4759aa870947/volumes"
Jul 29 19:54:17.651322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.651291    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"]
Jul 29 19:54:17.651859 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.651709    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b752b8e7-893d-4840-9516-4759aa870947" containerName="istio-proxy"
Jul 29 19:54:17.651859 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.651722    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b752b8e7-893d-4840-9516-4759aa870947" containerName="istio-proxy"
Jul 29 19:54:17.651859 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.651730    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b752b8e7-893d-4840-9516-4759aa870947" containerName="istio-proxy"
Jul 29 19:54:17.651859 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.651736    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b752b8e7-893d-4840-9516-4759aa870947" containerName="istio-proxy"
Jul 29 19:54:17.651859 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.651748    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" containerName="main"
Jul 29 19:54:17.651859 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.651754    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" containerName="main"
Jul 29 19:54:17.651859 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.651802    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b752b8e7-893d-4840-9516-4759aa870947" containerName="istio-proxy"
Jul 29 19:54:17.651859 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.651810    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb1292f5-2a7b-4290-85e8-d9f39c31b8b1" containerName="main"
Jul 29 19:54:17.656322 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.656306    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.659308 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.659287    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-59d5db-openshift-default-dockercfg-7sm6n\""
Jul 29 19:54:17.659433 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.659293    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 19:54:17.669342 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.669318    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"]
Jul 29 19:54:17.750250 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.750219    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-data\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.750250 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.750252    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-certs\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.750440 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.750273    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-token\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.750440 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.750292    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-socket\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.750440 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.750309    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istiod-ca-cert\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.750440 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.750374    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqxs8\" (UniqueName: \"kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-kube-api-access-kqxs8\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.750598 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.750439    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-podinfo\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.750598 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.750560    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-credential-socket\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.750706 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.750597    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-envoy\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.851790 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.851752    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-podinfo\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.851940 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.851852    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-credential-socket\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.851940 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.851884    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-envoy\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.851940 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.851917    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-data\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.852071 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.851943    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-certs\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.852071 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.851973    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-token\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.852071 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.852005    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-socket\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.852071 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.852034    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istiod-ca-cert\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.852226 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.852074    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqxs8\" (UniqueName: \"kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-kube-api-access-kqxs8\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.852350 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.852326    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-data\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.852422 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.852388    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-credential-socket\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.852496 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.852429    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-certs\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.852563 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.852541    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-socket\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.852963 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.852944    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istiod-ca-cert\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.854417 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.854392    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-podinfo\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.854623 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.854606    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-envoy\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.866030 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.866009    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqxs8\" (UniqueName: \"kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-kube-api-access-kqxs8\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.866578 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.866553    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-token\") pod \"e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") " pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:17.967078 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:17.967047    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:18.102229 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:18.102204    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"]
Jul 29 19:54:18.103789 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:54:18.103763    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce9812ba_c191_47aa_8ffa_0cc5dc68abc7.slice/crio-fd4cb15db816aad69165f31738f830d8b1c375d931a1aebd0a3cc7ecc84bca0c WatchSource:0}: Error finding container fd4cb15db816aad69165f31738f830d8b1c375d931a1aebd0a3cc7ecc84bca0c: Status 404 returned error can't find the container with id fd4cb15db816aad69165f31738f830d8b1c375d931a1aebd0a3cc7ecc84bca0c
Jul 29 19:54:18.105790 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:18.105757    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:54:18.105871 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:18.105817    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:54:18.105871 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:18.105862    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:54:18.883567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:18.883530    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" event={"ID":"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7","Type":"ContainerStarted","Data":"03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1"}
Jul 29 19:54:18.883998 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:18.883592    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" event={"ID":"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7","Type":"ContainerStarted","Data":"fd4cb15db816aad69165f31738f830d8b1c375d931a1aebd0a3cc7ecc84bca0c"}
Jul 29 19:54:18.905462 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:18.905402    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" podStartSLOduration=1.9053875489999998 podStartE2EDuration="1.905387549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:54:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:54:18.903707388 +0000 UTC m=+2224.399468756" watchObservedRunningTime="2026-07-29 19:54:18.905387549 +0000 UTC m=+2224.401148891"
Jul 29 19:54:18.968056 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:18.968027    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:19.973084 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:19.973051    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:20.894123 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:20.894092    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:20.895282 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:20.895258    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:33.963416 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:33.963380    2566 generic.go:356] "Generic (PLEG): container finished" podID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerID="03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1" exitCode=0
Jul 29 19:54:33.963860 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:33.963447    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" event={"ID":"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7","Type":"ContainerDied","Data":"03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1"}
Jul 29 19:54:33.963860 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:33.963842    2566 scope.go:117] "RemoveContainer" containerID="03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1"
Jul 29 19:54:33.964844 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:33.964813    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:54:33.964958 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:33.964900    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:54:33.964958 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:33.964948    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:54:34.971334 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:34.971291    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" event={"ID":"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7","Type":"ContainerStarted","Data":"9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91"}
Jul 29 19:54:35.894807 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:35.894772    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:35.967250 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:35.967210    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:35.968419 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:35.968392    2566 patch_prober.go:28] interesting pod/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 29 19:54:35.968583 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:35.968469    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 29 19:54:36.967513 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:36.967421    2566 patch_prober.go:28] interesting pod/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 29 19:54:36.967851 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:36.967510    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 29 19:54:37.968178 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:37.968141    2566 patch_prober.go:28] interesting pod/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 29 19:54:37.968617 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:37.968198    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 29 19:54:38.967923 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:38.967888    2566 patch_prober.go:28] interesting pod/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 29 19:54:38.968102 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:38.967942    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 29 19:54:39.971469 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:39.971424    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:39.972384 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:39.972367    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:54:49.958360 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:49.958326    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"]
Jul 29 19:54:49.958820 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:49.958794    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b752b8e7-893d-4840-9516-4759aa870947" containerName="istio-proxy"
Jul 29 19:54:49.960847 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:49.960832    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:54:49.963294 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:49.963270    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-accf7d-dockercfg-ljxch\""
Jul 29 19:54:49.963294 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:49.963284    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-accf7d\""
Jul 29 19:54:49.970715 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:49.970690    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"]
Jul 29 19:54:50.073950 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:50.073914    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1edb744-054c-4256-8854-6f9ae9e4c586-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp\" (UID: \"a1edb744-054c-4256-8854-6f9ae9e4c586\") " pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:54:50.073950 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:50.073953    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8sl9\" (UniqueName: \"kubernetes.io/projected/a1edb744-054c-4256-8854-6f9ae9e4c586-kube-api-access-d8sl9\") pod \"payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp\" (UID: \"a1edb744-054c-4256-8854-6f9ae9e4c586\") " pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:54:50.174937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:50.174905    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1edb744-054c-4256-8854-6f9ae9e4c586-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp\" (UID: \"a1edb744-054c-4256-8854-6f9ae9e4c586\") " pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:54:50.174937 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:50.174942    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8sl9\" (UniqueName: \"kubernetes.io/projected/a1edb744-054c-4256-8854-6f9ae9e4c586-kube-api-access-d8sl9\") pod \"payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp\" (UID: \"a1edb744-054c-4256-8854-6f9ae9e4c586\") " pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:54:50.175595 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:50.175571    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1edb744-054c-4256-8854-6f9ae9e4c586-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp\" (UID: \"a1edb744-054c-4256-8854-6f9ae9e4c586\") " pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:54:50.183756 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:50.183734    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8sl9\" (UniqueName: \"kubernetes.io/projected/a1edb744-054c-4256-8854-6f9ae9e4c586-kube-api-access-d8sl9\") pod \"payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp\" (UID: \"a1edb744-054c-4256-8854-6f9ae9e4c586\") " pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:54:50.272193 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:50.272132    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:54:50.411821 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:50.411795    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"]
Jul 29 19:54:50.413959 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:54:50.413914    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1edb744_054c_4256_8854_6f9ae9e4c586.slice/crio-30658a23b2947ee5494152e15917239c2ecc2bd8bdec614655d0daff751d7722 WatchSource:0}: Error finding container 30658a23b2947ee5494152e15917239c2ecc2bd8bdec614655d0daff751d7722: Status 404 returned error can't find the container with id 30658a23b2947ee5494152e15917239c2ecc2bd8bdec614655d0daff751d7722
Jul 29 19:54:51.050125 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:51.050085    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp" event={"ID":"a1edb744-054c-4256-8854-6f9ae9e4c586","Type":"ContainerStarted","Data":"adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823"}
Jul 29 19:54:51.050630 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:51.050138    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp" event={"ID":"a1edb744-054c-4256-8854-6f9ae9e4c586","Type":"ContainerStarted","Data":"30658a23b2947ee5494152e15917239c2ecc2bd8bdec614655d0daff751d7722"}
Jul 29 19:54:51.050630 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:51.050161    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:54:51.068208 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:54:51.068159    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp" podStartSLOduration=2.06814633 podStartE2EDuration="2.06814633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:54:51.066966885 +0000 UTC m=+2256.562728253" watchObservedRunningTime="2026-07-29 19:54:51.06814633 +0000 UTC m=+2256.563907670"
Jul 29 19:55:02.057659 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:02.057627    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:55:51.071792 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.071757    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"]
Jul 29 19:55:51.075494 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.075476    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.079650 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.079625    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-accf7d\"/\"openshift-service-ca.crt\""
Jul 29 19:55:51.079768 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.079654    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-accf7d\"/\"kube-root-ca.crt\""
Jul 29 19:55:51.079768 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.079631    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-accf7d\"/\"default-dockercfg-x8qz4\""
Jul 29 19:55:51.079768 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.079654    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-accf7d\"/\"test-model-accf7d-kserve-self-signed-certs\""
Jul 29 19:55:51.085721 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.085698    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"]
Jul 29 19:55:51.162152 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.162112    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-dshm\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.162340 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.162209    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de997424-a5df-4101-a7f3-7ffe8679e87e-tls-certs\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.162340 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.162240    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bj82\" (UniqueName: \"kubernetes.io/projected/de997424-a5df-4101-a7f3-7ffe8679e87e-kube-api-access-9bj82\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.162340 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.162330    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-tmp-dir\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.162575 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.162440    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-model-cache\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.162575 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.162542    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-home\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.263683 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.263639    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de997424-a5df-4101-a7f3-7ffe8679e87e-tls-certs\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.263683 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.263685    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bj82\" (UniqueName: \"kubernetes.io/projected/de997424-a5df-4101-a7f3-7ffe8679e87e-kube-api-access-9bj82\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.263957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.263722    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-tmp-dir\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.263957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.263771    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-model-cache\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.263957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.263789    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-home\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.263957 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.263817    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-dshm\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.264214 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.264187    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-model-cache\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.264277 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.264241    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-home\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.264311 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.264273    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-tmp-dir\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.266095 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.266073    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-dshm\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.266285 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.266271    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de997424-a5df-4101-a7f3-7ffe8679e87e-tls-certs\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.273189 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.273164    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bj82\" (UniqueName: \"kubernetes.io/projected/de997424-a5df-4101-a7f3-7ffe8679e87e-kube-api-access-9bj82\") pod \"test-model-accf7d-kserve-78d695d5f7-cdcr2\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") " pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.385801 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.385706    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:55:51.525858 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:51.525831    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"]
Jul 29 19:55:51.527545 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:55:51.527517    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde997424_a5df_4101_a7f3_7ffe8679e87e.slice/crio-6375e1d6c432cdc9390eba59f1d29903e1266f0a10efe182998e5cb67c386213 WatchSource:0}: Error finding container 6375e1d6c432cdc9390eba59f1d29903e1266f0a10efe182998e5cb67c386213: Status 404 returned error can't find the container with id 6375e1d6c432cdc9390eba59f1d29903e1266f0a10efe182998e5cb67c386213
Jul 29 19:55:52.363041 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:52.363003    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2" event={"ID":"de997424-a5df-4101-a7f3-7ffe8679e87e","Type":"ContainerStarted","Data":"d2c69e8725bb4567d406fdbd856da560f0475a4a9fdff504d3150cdda61c5e96"}
Jul 29 19:55:52.363406 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:52.363051    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2" event={"ID":"de997424-a5df-4101-a7f3-7ffe8679e87e","Type":"ContainerStarted","Data":"6375e1d6c432cdc9390eba59f1d29903e1266f0a10efe182998e5cb67c386213"}
Jul 29 19:55:52.382995 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:55:52.382941    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2" podStartSLOduration=1.382924899 podStartE2EDuration="1.382924899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:55:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:55:52.381143375 +0000 UTC m=+2317.876904720" watchObservedRunningTime="2026-07-29 19:55:52.382924899 +0000 UTC m=+2317.878686241"
Jul 29 19:56:01.386378 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:01.386338    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:56:01.386898 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:01.386392    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:56:01.398677 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:01.398648    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:56:01.421393 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:01.421369    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:56:44.541241 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:44.541199    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"]
Jul 29 19:56:59.560588 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.560546    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2" podUID="de997424-a5df-4101-a7f3-7ffe8679e87e" containerName="main" containerID="cri-o://d2c69e8725bb4567d406fdbd856da560f0475a4a9fdff504d3150cdda61c5e96" gracePeriod=45
Jul 29 19:56:59.703964 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.703934    2566 generic.go:356] "Generic (PLEG): container finished" podID="de997424-a5df-4101-a7f3-7ffe8679e87e" containerID="d2c69e8725bb4567d406fdbd856da560f0475a4a9fdff504d3150cdda61c5e96" exitCode=0
Jul 29 19:56:59.704096 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.704002    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2" event={"ID":"de997424-a5df-4101-a7f3-7ffe8679e87e","Type":"ContainerDied","Data":"d2c69e8725bb4567d406fdbd856da560f0475a4a9fdff504d3150cdda61c5e96"}
Jul 29 19:56:59.804818 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.804796    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:56:59.908992 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.908921    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-dshm\") pod \"de997424-a5df-4101-a7f3-7ffe8679e87e\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") "
Jul 29 19:56:59.908992 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.908952    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de997424-a5df-4101-a7f3-7ffe8679e87e-tls-certs\") pod \"de997424-a5df-4101-a7f3-7ffe8679e87e\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") "
Jul 29 19:56:59.908992 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.908969    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-tmp-dir\") pod \"de997424-a5df-4101-a7f3-7ffe8679e87e\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") "
Jul 29 19:56:59.908992 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.908989    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bj82\" (UniqueName: \"kubernetes.io/projected/de997424-a5df-4101-a7f3-7ffe8679e87e-kube-api-access-9bj82\") pod \"de997424-a5df-4101-a7f3-7ffe8679e87e\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") "
Jul 29 19:56:59.909307 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.909079    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-home\") pod \"de997424-a5df-4101-a7f3-7ffe8679e87e\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") "
Jul 29 19:56:59.909307 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.909121    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-model-cache\") pod \"de997424-a5df-4101-a7f3-7ffe8679e87e\" (UID: \"de997424-a5df-4101-a7f3-7ffe8679e87e\") "
Jul 29 19:56:59.909307 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.909269    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de997424-a5df-4101-a7f3-7ffe8679e87e" (UID: "de997424-a5df-4101-a7f3-7ffe8679e87e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:56:59.909482 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.909331    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-home" (OuterVolumeSpecName: "home") pod "de997424-a5df-4101-a7f3-7ffe8679e87e" (UID: "de997424-a5df-4101-a7f3-7ffe8679e87e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:56:59.909482 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.909416    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-model-cache" (OuterVolumeSpecName: "model-cache") pod "de997424-a5df-4101-a7f3-7ffe8679e87e" (UID: "de997424-a5df-4101-a7f3-7ffe8679e87e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:56:59.909482 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.909435    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-home\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:56:59.909607 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.909480    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-tmp-dir\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:56:59.911076 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.911052    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-dshm" (OuterVolumeSpecName: "dshm") pod "de997424-a5df-4101-a7f3-7ffe8679e87e" (UID: "de997424-a5df-4101-a7f3-7ffe8679e87e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:56:59.911187 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.911137    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de997424-a5df-4101-a7f3-7ffe8679e87e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de997424-a5df-4101-a7f3-7ffe8679e87e" (UID: "de997424-a5df-4101-a7f3-7ffe8679e87e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:56:59.911245 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:56:59.911186    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de997424-a5df-4101-a7f3-7ffe8679e87e-kube-api-access-9bj82" (OuterVolumeSpecName: "kube-api-access-9bj82") pod "de997424-a5df-4101-a7f3-7ffe8679e87e" (UID: "de997424-a5df-4101-a7f3-7ffe8679e87e"). InnerVolumeSpecName "kube-api-access-9bj82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:57:00.010624 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:00.010593    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-model-cache\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:00.010624 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:00.010620    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de997424-a5df-4101-a7f3-7ffe8679e87e-dshm\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:00.010624 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:00.010628    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de997424-a5df-4101-a7f3-7ffe8679e87e-tls-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:00.010825 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:00.010637    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bj82\" (UniqueName: \"kubernetes.io/projected/de997424-a5df-4101-a7f3-7ffe8679e87e-kube-api-access-9bj82\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:00.715903 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:00.715874    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"
Jul 29 19:57:00.716360 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:00.715865    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2" event={"ID":"de997424-a5df-4101-a7f3-7ffe8679e87e","Type":"ContainerDied","Data":"6375e1d6c432cdc9390eba59f1d29903e1266f0a10efe182998e5cb67c386213"}
Jul 29 19:57:00.716360 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:00.715999    2566 scope.go:117] "RemoveContainer" containerID="d2c69e8725bb4567d406fdbd856da560f0475a4a9fdff504d3150cdda61c5e96"
Jul 29 19:57:00.740509 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:00.740476    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"]
Jul 29 19:57:00.746305 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:00.746284    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-accf7d/test-model-accf7d-kserve-78d695d5f7-cdcr2"]
Jul 29 19:57:01.102532 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:01.102436    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de997424-a5df-4101-a7f3-7ffe8679e87e" path="/var/lib/kubelet/pods/de997424-a5df-4101-a7f3-7ffe8679e87e/volumes"
Jul 29 19:57:02.135504 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:02.135471    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"]
Jul 29 19:57:02.135910 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:02.135737    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerName="istio-proxy" containerID="cri-o://9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91" gracePeriod=30
Jul 29 19:57:07.385822 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.385799    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:57:07.386685 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.386666    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqxs8\" (UniqueName: \"kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-kube-api-access-kqxs8\") pod \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") "
Jul 29 19:57:07.386742 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.386706    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-envoy\") pod \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") "
Jul 29 19:57:07.386742 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.386733    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-socket\") pod \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") "
Jul 29 19:57:07.386818 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.386755    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-token\") pod \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") "
Jul 29 19:57:07.386818 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.386774    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-certs\") pod \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") "
Jul 29 19:57:07.386818 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.386790    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-podinfo\") pod \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") "
Jul 29 19:57:07.386971 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.386835    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istiod-ca-cert\") pod \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") "
Jul 29 19:57:07.386971 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.386865    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-credential-socket\") pod \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") "
Jul 29 19:57:07.386971 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.386893    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-data\") pod \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\" (UID: \"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7\") "
Jul 29 19:57:07.387120 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.387086    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" (UID: "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:57:07.387120 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.387098    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" (UID: "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:57:07.387487 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.387315    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:07.387487 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.387349    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-workload-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:07.387487 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.387314    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" (UID: "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:57:07.387487 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.387442    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" (UID: "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:57:07.387825 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.387803    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-data" (OuterVolumeSpecName: "istio-data") pod "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" (UID: "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:57:07.388916 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.388894    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" (UID: "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:57:07.389252 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.389231    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" (UID: "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 19:57:07.389331 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.389277    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-kube-api-access-kqxs8" (OuterVolumeSpecName: "kube-api-access-kqxs8") pod "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" (UID: "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7"). InnerVolumeSpecName "kube-api-access-kqxs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:57:07.389331 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.389305    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-token" (OuterVolumeSpecName: "istio-token") pod "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" (UID: "ce9812ba-c191-47aa-8ffa-0cc5dc68abc7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:57:07.487997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.487965    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istiod-ca-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:07.487997 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.487995    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-credential-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:07.488207 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.488005    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-data\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:07.488207 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.488015    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqxs8\" (UniqueName: \"kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-kube-api-access-kqxs8\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:07.488207 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.488023    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-envoy\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:07.488207 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.488031    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-token\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:07.488207 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.488040    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7-istio-podinfo\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:07.754230 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.754135    2566 generic.go:356] "Generic (PLEG): container finished" podID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerID="9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91" exitCode=0
Jul 29 19:57:07.754230 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.754192    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"
Jul 29 19:57:07.754495 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.754233    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" event={"ID":"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7","Type":"ContainerDied","Data":"9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91"}
Jul 29 19:57:07.754495 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.754302    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k" event={"ID":"ce9812ba-c191-47aa-8ffa-0cc5dc68abc7","Type":"ContainerDied","Data":"fd4cb15db816aad69165f31738f830d8b1c375d931a1aebd0a3cc7ecc84bca0c"}
Jul 29 19:57:07.754495 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.754335    2566 scope.go:117] "RemoveContainer" containerID="9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91"
Jul 29 19:57:07.763566 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.763548    2566 scope.go:117] "RemoveContainer" containerID="03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1"
Jul 29 19:57:07.771292 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.771274    2566 scope.go:117] "RemoveContainer" containerID="9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91"
Jul 29 19:57:07.771588 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:57:07.771564    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91\": container with ID starting with 9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91 not found: ID does not exist" containerID="9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91"
Jul 29 19:57:07.771696 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.771596    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91"} err="failed to get container status \"9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91\": rpc error: code = NotFound desc = could not find container \"9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91\": container with ID starting with 9ea38a5291a55db6e913767d4fe07ae2a603d2dcb4e63b05d7ca279b8353ea91 not found: ID does not exist"
Jul 29 19:57:07.771696 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.771614    2566 scope.go:117] "RemoveContainer" containerID="03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1"
Jul 29 19:57:07.771861 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:57:07.771843    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1\": container with ID starting with 03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1 not found: ID does not exist" containerID="03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1"
Jul 29 19:57:07.771902 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.771867    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1"} err="failed to get container status \"03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1\": rpc error: code = NotFound desc = could not find container \"03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1\": container with ID starting with 03a7fd2f29e5f0cd79f899a1ac27318bc725131a02a4f9b56ce964406978e7e1 not found: ID does not exist"
Jul 29 19:57:07.776272 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.776251    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"]
Jul 29 19:57:07.782258 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:07.782237    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-59d5db-openshift-default-556766f6f6-nrn2k"]
Jul 29 19:57:09.102325 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:09.102293    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" path="/var/lib/kubelet/pods/ce9812ba-c191-47aa-8ffa-0cc5dc68abc7/volumes"
Jul 29 19:57:13.046397 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.046360    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"]
Jul 29 19:57:13.046878 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.046589    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp" podUID="a1edb744-054c-4256-8854-6f9ae9e4c586" containerName="payload-processing" containerID="cri-o://adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823" gracePeriod=30
Jul 29 19:57:13.302058 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.302002    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:57:13.342717 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.342682    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8sl9\" (UniqueName: \"kubernetes.io/projected/a1edb744-054c-4256-8854-6f9ae9e4c586-kube-api-access-d8sl9\") pod \"a1edb744-054c-4256-8854-6f9ae9e4c586\" (UID: \"a1edb744-054c-4256-8854-6f9ae9e4c586\") "
Jul 29 19:57:13.342866 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.342792    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1edb744-054c-4256-8854-6f9ae9e4c586-plugins-config-volume\") pod \"a1edb744-054c-4256-8854-6f9ae9e4c586\" (UID: \"a1edb744-054c-4256-8854-6f9ae9e4c586\") "
Jul 29 19:57:13.343259 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.343227    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1edb744-054c-4256-8854-6f9ae9e4c586-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1edb744-054c-4256-8854-6f9ae9e4c586" (UID: "a1edb744-054c-4256-8854-6f9ae9e4c586"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:57:13.344926 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.344898    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1edb744-054c-4256-8854-6f9ae9e4c586-kube-api-access-d8sl9" (OuterVolumeSpecName: "kube-api-access-d8sl9") pod "a1edb744-054c-4256-8854-6f9ae9e4c586" (UID: "a1edb744-054c-4256-8854-6f9ae9e4c586"). InnerVolumeSpecName "kube-api-access-d8sl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:57:13.443692 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.443656    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1edb744-054c-4256-8854-6f9ae9e4c586-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:13.443692 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.443689    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8sl9\" (UniqueName: \"kubernetes.io/projected/a1edb744-054c-4256-8854-6f9ae9e4c586-kube-api-access-d8sl9\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:57:13.784237 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.784208    2566 generic.go:356] "Generic (PLEG): container finished" podID="a1edb744-054c-4256-8854-6f9ae9e4c586" containerID="adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823" exitCode=0
Jul 29 19:57:13.784421 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.784257    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"
Jul 29 19:57:13.784421 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.784262    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp" event={"ID":"a1edb744-054c-4256-8854-6f9ae9e4c586","Type":"ContainerDied","Data":"adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823"}
Jul 29 19:57:13.784421 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.784362    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp" event={"ID":"a1edb744-054c-4256-8854-6f9ae9e4c586","Type":"ContainerDied","Data":"30658a23b2947ee5494152e15917239c2ecc2bd8bdec614655d0daff751d7722"}
Jul 29 19:57:13.784421 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.784384    2566 scope.go:117] "RemoveContainer" containerID="adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823"
Jul 29 19:57:13.793177 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.793159    2566 scope.go:117] "RemoveContainer" containerID="adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823"
Jul 29 19:57:13.793425 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:57:13.793403    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823\": container with ID starting with adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823 not found: ID does not exist" containerID="adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823"
Jul 29 19:57:13.793493 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.793433    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823"} err="failed to get container status \"adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823\": rpc error: code = NotFound desc = could not find container \"adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823\": container with ID starting with adf827ebc046137cea5aed8e75a2c8226ab1d48a7ae40c2374a74fd28f6f5823 not found: ID does not exist"
Jul 29 19:57:13.807290 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.807253    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"]
Jul 29 19:57:13.811094 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:13.811070    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-accf7d-6b5cfb689b-rv6dp"]
Jul 29 19:57:15.103138 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:15.103107    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1edb744-054c-4256-8854-6f9ae9e4c586" path="/var/lib/kubelet/pods/a1edb744-054c-4256-8854-6f9ae9e4c586/volumes"
Jul 29 19:57:20.593986 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.593938    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"]
Jul 29 19:57:20.594554 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.594436    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerName="istio-proxy"
Jul 29 19:57:20.594554 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.594466    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerName="istio-proxy"
Jul 29 19:57:20.594554 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.594477    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de997424-a5df-4101-a7f3-7ffe8679e87e" containerName="main"
Jul 29 19:57:20.594554 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.594483    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="de997424-a5df-4101-a7f3-7ffe8679e87e" containerName="main"
Jul 29 19:57:20.594554 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.594493    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1edb744-054c-4256-8854-6f9ae9e4c586" containerName="payload-processing"
Jul 29 19:57:20.594554 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.594498    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1edb744-054c-4256-8854-6f9ae9e4c586" containerName="payload-processing"
Jul 29 19:57:20.594874 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.594565    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerName="istio-proxy"
Jul 29 19:57:20.594874 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.594573    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="de997424-a5df-4101-a7f3-7ffe8679e87e" containerName="main"
Jul 29 19:57:20.594874 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.594582    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1edb744-054c-4256-8854-6f9ae9e4c586" containerName="payload-processing"
Jul 29 19:57:20.597565 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.597540    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.600700 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.600677    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 19:57:20.600700 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.600696    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-dc9ec3-openshift-default-dockercfg-9ktqj\""
Jul 29 19:57:20.610077 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.610055    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"]
Jul 29 19:57:20.712500 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.712461    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istiod-ca-cert\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.712698 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.712514    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-podinfo\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.712698 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.712551    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-data\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.712698 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.712637    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqszj\" (UniqueName: \"kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-kube-api-access-sqszj\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.712876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.712707    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-socket\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.712876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.712772    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-credential-socket\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.712876 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.712815    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-envoy\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.712989 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.712888    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-token\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.712989 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.712966    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-certs\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814051 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814018    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-certs\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814222 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814059    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istiod-ca-cert\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814222 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814077    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-podinfo\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814222 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814103    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-data\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814222 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814162    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqszj\" (UniqueName: \"kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-kube-api-access-sqszj\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814222 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814191    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-socket\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814495 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814225    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-credential-socket\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814565 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814490    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-certs\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814565 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814523    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-data\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814672 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814560    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-envoy\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814672 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814563    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-socket\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814672 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814601    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-token\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.814672 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.814611    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-credential-socket\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.815160 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.815144    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istiod-ca-cert\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.816983 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.816960    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-envoy\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.817284 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.817264    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-podinfo\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.822764 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.822744    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-token\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.822906 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.822887    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqszj\" (UniqueName: \"kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-kube-api-access-sqszj\") pod \"e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") " pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:20.907146 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:20.907070    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:21.060279 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:21.060254    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"]
Jul 29 19:57:21.062274 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:57:21.062246    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod592e40c3_8f7a_4007_87c4_62ab5b0d48fd.slice/crio-e79a494ba8dd822e23e7695b62256893a7b6696b48e992f7733d39f76667ca07 WatchSource:0}: Error finding container e79a494ba8dd822e23e7695b62256893a7b6696b48e992f7733d39f76667ca07: Status 404 returned error can't find the container with id e79a494ba8dd822e23e7695b62256893a7b6696b48e992f7733d39f76667ca07
Jul 29 19:57:21.064503 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:21.064468    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:57:21.064585 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:21.064549    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:57:21.064632 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:21.064591    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:57:21.826644 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:21.826611    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" event={"ID":"592e40c3-8f7a-4007-87c4-62ab5b0d48fd","Type":"ContainerStarted","Data":"df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd"}
Jul 29 19:57:21.827004 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:21.826662    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" event={"ID":"592e40c3-8f7a-4007-87c4-62ab5b0d48fd","Type":"ContainerStarted","Data":"e79a494ba8dd822e23e7695b62256893a7b6696b48e992f7733d39f76667ca07"}
Jul 29 19:57:21.852588 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:21.852541    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" podStartSLOduration=1.852525311 podStartE2EDuration="1.852525311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:57:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:57:21.850447081 +0000 UTC m=+2407.346208450" watchObservedRunningTime="2026-07-29 19:57:21.852525311 +0000 UTC m=+2407.348286656"
Jul 29 19:57:21.907581 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:21.907555    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:22.912844 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:22.912816    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:23.837731 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:23.837700    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:23.838642 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:23.838619    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:38.838593 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:38.838495    2566 patch_prober.go:28] interesting pod/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 29 19:57:38.838593 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:38.838577    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 29 19:57:38.914933 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:38.914902    2566 generic.go:356] "Generic (PLEG): container finished" podID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerID="df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd" exitCode=0
Jul 29 19:57:38.915103 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:38.914975    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" event={"ID":"592e40c3-8f7a-4007-87c4-62ab5b0d48fd","Type":"ContainerDied","Data":"df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd"}
Jul 29 19:57:38.915420 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:38.915403    2566 scope.go:117] "RemoveContainer" containerID="df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd"
Jul 29 19:57:38.916435 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:38.916410    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:57:38.916554 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:38.916489    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:57:38.916554 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:38.916522    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:57:39.921674 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:39.921638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" event={"ID":"592e40c3-8f7a-4007-87c4-62ab5b0d48fd","Type":"ContainerStarted","Data":"35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9"}
Jul 29 19:57:40.908287 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:40.908250    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:40.908514 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:40.908490    2566 patch_prober.go:28] interesting pod/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 29 19:57:40.908576 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:40.908538    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 29 19:57:41.908292 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:41.908260    2566 patch_prober.go:28] interesting pod/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 29 19:57:41.908792 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:41.908351    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 29 19:57:42.907875 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:42.907838    2566 patch_prober.go:28] interesting pod/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 29 19:57:42.908053 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:42.907907    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 29 19:57:43.911428 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:43.911397    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:43.911874 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:43.911687    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:43.912521 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:43.912500    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:57:52.960959 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:52.960926    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"]
Jul 29 19:57:52.961445 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:52.961397    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerName="istio-proxy"
Jul 29 19:57:52.961445 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:52.961413    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerName="istio-proxy"
Jul 29 19:57:52.961548 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:52.961489    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce9812ba-c191-47aa-8ffa-0cc5dc68abc7" containerName="istio-proxy"
Jul 29 19:57:52.964760 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:52.964739    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:57:52.967274 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:52.967249    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0e0fe5\""
Jul 29 19:57:52.975616 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:52.975594    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"]
Jul 29 19:57:53.010577 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.010545    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"]
Jul 29 19:57:53.014040 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.014020    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:57:53.016840 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.016819    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0e0fe5-dockercfg-wsw9v\""
Jul 29 19:57:53.022623 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.022598    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"]
Jul 29 19:57:53.034705 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.034680    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32fe5371-0042-4ac5-a6ea-08018ce7147c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt\" (UID: \"32fe5371-0042-4ac5-a6ea-08018ce7147c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:57:53.034840 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.034821    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsfnk\" (UniqueName: \"kubernetes.io/projected/32fe5371-0042-4ac5-a6ea-08018ce7147c-kube-api-access-lsfnk\") pod \"payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt\" (UID: \"32fe5371-0042-4ac5-a6ea-08018ce7147c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:57:53.135327 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.135297    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32fe5371-0042-4ac5-a6ea-08018ce7147c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt\" (UID: \"32fe5371-0042-4ac5-a6ea-08018ce7147c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:57:53.135538 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.135344    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a76e4ddf-9575-4985-b154-ecf82e514d57-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8\" (UID: \"a76e4ddf-9575-4985-b154-ecf82e514d57\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:57:53.135538 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.135377    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnpnz\" (UniqueName: \"kubernetes.io/projected/a76e4ddf-9575-4985-b154-ecf82e514d57-kube-api-access-gnpnz\") pod \"payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8\" (UID: \"a76e4ddf-9575-4985-b154-ecf82e514d57\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:57:53.135670 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.135629    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsfnk\" (UniqueName: \"kubernetes.io/projected/32fe5371-0042-4ac5-a6ea-08018ce7147c-kube-api-access-lsfnk\") pod \"payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt\" (UID: \"32fe5371-0042-4ac5-a6ea-08018ce7147c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:57:53.136240 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.136214    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32fe5371-0042-4ac5-a6ea-08018ce7147c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt\" (UID: \"32fe5371-0042-4ac5-a6ea-08018ce7147c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:57:53.144036 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.144010    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsfnk\" (UniqueName: \"kubernetes.io/projected/32fe5371-0042-4ac5-a6ea-08018ce7147c-kube-api-access-lsfnk\") pod \"payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt\" (UID: \"32fe5371-0042-4ac5-a6ea-08018ce7147c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:57:53.237099 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.237016    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a76e4ddf-9575-4985-b154-ecf82e514d57-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8\" (UID: \"a76e4ddf-9575-4985-b154-ecf82e514d57\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:57:53.237099 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.237058    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnpnz\" (UniqueName: \"kubernetes.io/projected/a76e4ddf-9575-4985-b154-ecf82e514d57-kube-api-access-gnpnz\") pod \"payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8\" (UID: \"a76e4ddf-9575-4985-b154-ecf82e514d57\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:57:53.237961 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.237939    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a76e4ddf-9575-4985-b154-ecf82e514d57-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8\" (UID: \"a76e4ddf-9575-4985-b154-ecf82e514d57\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:57:53.245323 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.245301    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnpnz\" (UniqueName: \"kubernetes.io/projected/a76e4ddf-9575-4985-b154-ecf82e514d57-kube-api-access-gnpnz\") pod \"payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8\" (UID: \"a76e4ddf-9575-4985-b154-ecf82e514d57\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:57:53.276147 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.276124    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:57:53.324182 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.324150    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:57:53.429808 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.429784    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"]
Jul 29 19:57:53.432521 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:57:53.432481    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32fe5371_0042_4ac5_a6ea_08018ce7147c.slice/crio-0527f74a7b33972ffece076e5b367f88d0da4aee0d6236d48a6d2145ffdbf913 WatchSource:0}: Error finding container 0527f74a7b33972ffece076e5b367f88d0da4aee0d6236d48a6d2145ffdbf913: Status 404 returned error can't find the container with id 0527f74a7b33972ffece076e5b367f88d0da4aee0d6236d48a6d2145ffdbf913
Jul 29 19:57:53.493152 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.493126    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"]
Jul 29 19:57:53.509561 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:57:53.509533    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda76e4ddf_9575_4985_b154_ecf82e514d57.slice/crio-c27dfe9db36dc8695ea81ff0afebc984e5bca300fd0cb9ef69ff1628bf4d30e1 WatchSource:0}: Error finding container c27dfe9db36dc8695ea81ff0afebc984e5bca300fd0cb9ef69ff1628bf4d30e1: Status 404 returned error can't find the container with id c27dfe9db36dc8695ea81ff0afebc984e5bca300fd0cb9ef69ff1628bf4d30e1
Jul 29 19:57:53.992284 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.992239    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt" event={"ID":"32fe5371-0042-4ac5-a6ea-08018ce7147c","Type":"ContainerStarted","Data":"7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a"}
Jul 29 19:57:53.992708 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.992309    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt" event={"ID":"32fe5371-0042-4ac5-a6ea-08018ce7147c","Type":"ContainerStarted","Data":"0527f74a7b33972ffece076e5b367f88d0da4aee0d6236d48a6d2145ffdbf913"}
Jul 29 19:57:53.992708 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.992337    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:57:53.993745 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.993714    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8" event={"ID":"a76e4ddf-9575-4985-b154-ecf82e514d57","Type":"ContainerStarted","Data":"bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5"}
Jul 29 19:57:53.993842 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.993766    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8" event={"ID":"a76e4ddf-9575-4985-b154-ecf82e514d57","Type":"ContainerStarted","Data":"c27dfe9db36dc8695ea81ff0afebc984e5bca300fd0cb9ef69ff1628bf4d30e1"}
Jul 29 19:57:53.993842 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:53.993792    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:57:54.009046 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:54.009001    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt" podStartSLOduration=2.008984102 podStartE2EDuration="2.008984102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:57:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:57:54.00891058 +0000 UTC m=+2439.504671945" watchObservedRunningTime="2026-07-29 19:57:54.008984102 +0000 UTC m=+2439.504745446"
Jul 29 19:57:54.025620 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:57:54.025575    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8" podStartSLOduration=2.025561792 podStartE2EDuration="2.025561792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:57:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:57:54.024264894 +0000 UTC m=+2439.520026236" watchObservedRunningTime="2026-07-29 19:57:54.025561792 +0000 UTC m=+2439.521323169"
Jul 29 19:58:05.001288 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:05.001255    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:58:05.001796 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:05.001333    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:58:42.452379 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:42.452346    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"]
Jul 29 19:58:42.453007 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:42.452555    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy" containerID="cri-o://35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9" gracePeriod=30
Jul 29 19:58:43.914141 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:43.914106    2566 patch_prober.go:28] interesting pod/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 19:58:43.914533 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:43.914177    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 19:58:47.711061 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.711037    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:58:47.863673 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.863597    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-podinfo\") pod \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") "
Jul 29 19:58:47.863673 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.863628    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-socket\") pod \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") "
Jul 29 19:58:47.863673 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.863669    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-certs\") pod \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") "
Jul 29 19:58:47.863879 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.863751    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-data\") pod \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") "
Jul 29 19:58:47.863879 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.863822    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-envoy\") pod \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") "
Jul 29 19:58:47.863879 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.863859    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-token\") pod \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") "
Jul 29 19:58:47.864040 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.863903    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "592e40c3-8f7a-4007-87c4-62ab5b0d48fd" (UID: "592e40c3-8f7a-4007-87c4-62ab5b0d48fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:58:47.864040 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.863922    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istiod-ca-cert\") pod \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") "
Jul 29 19:58:47.864040 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.863930    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "592e40c3-8f7a-4007-87c4-62ab5b0d48fd" (UID: "592e40c3-8f7a-4007-87c4-62ab5b0d48fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:58:47.864040 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.863987    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-credential-socket\") pod \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") "
Jul 29 19:58:47.864241 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.864221    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "592e40c3-8f7a-4007-87c4-62ab5b0d48fd" (UID: "592e40c3-8f7a-4007-87c4-62ab5b0d48fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:58:47.864284 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.864266    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqszj\" (UniqueName: \"kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-kube-api-access-sqszj\") pod \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\" (UID: \"592e40c3-8f7a-4007-87c4-62ab5b0d48fd\") "
Jul 29 19:58:47.864284 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.864268    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "592e40c3-8f7a-4007-87c4-62ab5b0d48fd" (UID: "592e40c3-8f7a-4007-87c4-62ab5b0d48fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:58:47.864655 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.864621    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-data" (OuterVolumeSpecName: "istio-data") pod "592e40c3-8f7a-4007-87c4-62ab5b0d48fd" (UID: "592e40c3-8f7a-4007-87c4-62ab5b0d48fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:58:47.864814 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.864742    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:47.864814 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.864762    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-workload-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:47.864814 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.864776    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-data\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:47.864814 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.864790    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istiod-ca-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:47.864814 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.864803    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-credential-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:47.865951 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.865923    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "592e40c3-8f7a-4007-87c4-62ab5b0d48fd" (UID: "592e40c3-8f7a-4007-87c4-62ab5b0d48fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:58:47.866069 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.866046    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "592e40c3-8f7a-4007-87c4-62ab5b0d48fd" (UID: "592e40c3-8f7a-4007-87c4-62ab5b0d48fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 19:58:47.866069 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.866054    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "592e40c3-8f7a-4007-87c4-62ab5b0d48fd" (UID: "592e40c3-8f7a-4007-87c4-62ab5b0d48fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:58:47.866189 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.866171    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-kube-api-access-sqszj" (OuterVolumeSpecName: "kube-api-access-sqszj") pod "592e40c3-8f7a-4007-87c4-62ab5b0d48fd" (UID: "592e40c3-8f7a-4007-87c4-62ab5b0d48fd"). InnerVolumeSpecName "kube-api-access-sqszj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:58:47.965341 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.965304    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-envoy\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:47.965341 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.965327    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-token\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:47.965341 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.965337    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqszj\" (UniqueName: \"kubernetes.io/projected/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-kube-api-access-sqszj\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:47.965341 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:47.965346    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/592e40c3-8f7a-4007-87c4-62ab5b0d48fd-istio-podinfo\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:48.263691 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.263659    2566 generic.go:356] "Generic (PLEG): container finished" podID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerID="35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9" exitCode=0
Jul 29 19:58:48.263886 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.263718    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"
Jul 29 19:58:48.263886 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.263719    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" event={"ID":"592e40c3-8f7a-4007-87c4-62ab5b0d48fd","Type":"ContainerDied","Data":"35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9"}
Jul 29 19:58:48.263886 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.263861    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6" event={"ID":"592e40c3-8f7a-4007-87c4-62ab5b0d48fd","Type":"ContainerDied","Data":"e79a494ba8dd822e23e7695b62256893a7b6696b48e992f7733d39f76667ca07"}
Jul 29 19:58:48.264042 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.263903    2566 scope.go:117] "RemoveContainer" containerID="35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9"
Jul 29 19:58:48.273614 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.273597    2566 scope.go:117] "RemoveContainer" containerID="df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd"
Jul 29 19:58:48.281173 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.281153    2566 scope.go:117] "RemoveContainer" containerID="35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9"
Jul 29 19:58:48.281401 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:58:48.281383    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9\": container with ID starting with 35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9 not found: ID does not exist" containerID="35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9"
Jul 29 19:58:48.281469 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.281409    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9"} err="failed to get container status \"35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9\": rpc error: code = NotFound desc = could not find container \"35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9\": container with ID starting with 35ab162360a91fb8ba0c5248f23f929a03799ba2b07af973b1cef21bf18353d9 not found: ID does not exist"
Jul 29 19:58:48.281469 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.281424    2566 scope.go:117] "RemoveContainer" containerID="df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd"
Jul 29 19:58:48.281682 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:58:48.281663    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd\": container with ID starting with df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd not found: ID does not exist" containerID="df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd"
Jul 29 19:58:48.281740 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.281691    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd"} err="failed to get container status \"df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd\": rpc error: code = NotFound desc = could not find container \"df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd\": container with ID starting with df8cf8d9937891ef9c0cf4aa53da5784a27fe4cae87d030a80933bbf0b27ffcd not found: ID does not exist"
Jul 29 19:58:48.288922 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.288899    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"]
Jul 29 19:58:48.293576 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:48.293555    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-dc9ec3-openshift-default-d6fc9c854-mdxk6"]
Jul 29 19:58:49.102469 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:49.102430    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" path="/var/lib/kubelet/pods/592e40c3-8f7a-4007-87c4-62ab5b0d48fd/volumes"
Jul 29 19:58:58.264237 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.264196    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"]
Jul 29 19:58:58.264748 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.264482    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8" podUID="a76e4ddf-9575-4985-b154-ecf82e514d57" containerName="payload-processing" containerID="cri-o://bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5" gracePeriod=30
Jul 29 19:58:58.268084 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.268059    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"]
Jul 29 19:58:58.268703 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.268633    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt" podUID="32fe5371-0042-4ac5-a6ea-08018ce7147c" containerName="payload-pre-processing" containerID="cri-o://7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a" gracePeriod=30
Jul 29 19:58:58.549885 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.549854    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:58:58.551902 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.551882    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:58:58.665139 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.665110    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a76e4ddf-9575-4985-b154-ecf82e514d57-plugins-config-volume\") pod \"a76e4ddf-9575-4985-b154-ecf82e514d57\" (UID: \"a76e4ddf-9575-4985-b154-ecf82e514d57\") "
Jul 29 19:58:58.665333 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.665194    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32fe5371-0042-4ac5-a6ea-08018ce7147c-plugins-config-volume\") pod \"32fe5371-0042-4ac5-a6ea-08018ce7147c\" (UID: \"32fe5371-0042-4ac5-a6ea-08018ce7147c\") "
Jul 29 19:58:58.665333 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.665215    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnpnz\" (UniqueName: \"kubernetes.io/projected/a76e4ddf-9575-4985-b154-ecf82e514d57-kube-api-access-gnpnz\") pod \"a76e4ddf-9575-4985-b154-ecf82e514d57\" (UID: \"a76e4ddf-9575-4985-b154-ecf82e514d57\") "
Jul 29 19:58:58.665333 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.665276    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsfnk\" (UniqueName: \"kubernetes.io/projected/32fe5371-0042-4ac5-a6ea-08018ce7147c-kube-api-access-lsfnk\") pod \"32fe5371-0042-4ac5-a6ea-08018ce7147c\" (UID: \"32fe5371-0042-4ac5-a6ea-08018ce7147c\") "
Jul 29 19:58:58.665607 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.665584    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a76e4ddf-9575-4985-b154-ecf82e514d57-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a76e4ddf-9575-4985-b154-ecf82e514d57" (UID: "a76e4ddf-9575-4985-b154-ecf82e514d57"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:58:58.665665 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.665582    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32fe5371-0042-4ac5-a6ea-08018ce7147c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32fe5371-0042-4ac5-a6ea-08018ce7147c" (UID: "32fe5371-0042-4ac5-a6ea-08018ce7147c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:58:58.667393 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.667369    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a76e4ddf-9575-4985-b154-ecf82e514d57-kube-api-access-gnpnz" (OuterVolumeSpecName: "kube-api-access-gnpnz") pod "a76e4ddf-9575-4985-b154-ecf82e514d57" (UID: "a76e4ddf-9575-4985-b154-ecf82e514d57"). InnerVolumeSpecName "kube-api-access-gnpnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:58:58.667505 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.667408    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32fe5371-0042-4ac5-a6ea-08018ce7147c-kube-api-access-lsfnk" (OuterVolumeSpecName: "kube-api-access-lsfnk") pod "32fe5371-0042-4ac5-a6ea-08018ce7147c" (UID: "32fe5371-0042-4ac5-a6ea-08018ce7147c"). InnerVolumeSpecName "kube-api-access-lsfnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:58:58.766763 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.766736    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnpnz\" (UniqueName: \"kubernetes.io/projected/a76e4ddf-9575-4985-b154-ecf82e514d57-kube-api-access-gnpnz\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:58.766763 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.766761    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsfnk\" (UniqueName: \"kubernetes.io/projected/32fe5371-0042-4ac5-a6ea-08018ce7147c-kube-api-access-lsfnk\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:58.766763 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.766770    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a76e4ddf-9575-4985-b154-ecf82e514d57-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:58.766964 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:58.766781    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32fe5371-0042-4ac5-a6ea-08018ce7147c-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 19:58:59.319385 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.319347    2566 generic.go:356] "Generic (PLEG): container finished" podID="a76e4ddf-9575-4985-b154-ecf82e514d57" containerID="bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5" exitCode=0
Jul 29 19:58:59.319776 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.319402    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"
Jul 29 19:58:59.319776 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.319437    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8" event={"ID":"a76e4ddf-9575-4985-b154-ecf82e514d57","Type":"ContainerDied","Data":"bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5"}
Jul 29 19:58:59.319776 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.319526    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8" event={"ID":"a76e4ddf-9575-4985-b154-ecf82e514d57","Type":"ContainerDied","Data":"c27dfe9db36dc8695ea81ff0afebc984e5bca300fd0cb9ef69ff1628bf4d30e1"}
Jul 29 19:58:59.319776 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.319560    2566 scope.go:117] "RemoveContainer" containerID="bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5"
Jul 29 19:58:59.320893 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.320770    2566 generic.go:356] "Generic (PLEG): container finished" podID="32fe5371-0042-4ac5-a6ea-08018ce7147c" containerID="7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a" exitCode=0
Jul 29 19:58:59.320893 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.320826    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"
Jul 29 19:58:59.320893 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.320862    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt" event={"ID":"32fe5371-0042-4ac5-a6ea-08018ce7147c","Type":"ContainerDied","Data":"7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a"}
Jul 29 19:58:59.321025 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.320899    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt" event={"ID":"32fe5371-0042-4ac5-a6ea-08018ce7147c","Type":"ContainerDied","Data":"0527f74a7b33972ffece076e5b367f88d0da4aee0d6236d48a6d2145ffdbf913"}
Jul 29 19:58:59.328161 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.328141    2566 scope.go:117] "RemoveContainer" containerID="bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5"
Jul 29 19:58:59.328406 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:58:59.328388    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5\": container with ID starting with bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5 not found: ID does not exist" containerID="bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5"
Jul 29 19:58:59.328493 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.328417    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5"} err="failed to get container status \"bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5\": rpc error: code = NotFound desc = could not find container \"bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5\": container with ID starting with bcb0758aae83ffe309ec25eb96791bd44e13e8c7fb13f80edc9c51951dbcadb5 not found: ID does not exist"
Jul 29 19:58:59.328493 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.328442    2566 scope.go:117] "RemoveContainer" containerID="7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a"
Jul 29 19:58:59.335740 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.335724    2566 scope.go:117] "RemoveContainer" containerID="7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a"
Jul 29 19:58:59.335979 ip-10-0-129-107 kubenswrapper[2566]: E0729 19:58:59.335962    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a\": container with ID starting with 7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a not found: ID does not exist" containerID="7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a"
Jul 29 19:58:59.336038 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.335984    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a"} err="failed to get container status \"7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a\": rpc error: code = NotFound desc = could not find container \"7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a\": container with ID starting with 7acdfebac02121a6ed3d559173219831d9b33d5587a065251bda7e002e2f9e0a not found: ID does not exist"
Jul 29 19:58:59.339613 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.339592    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"]
Jul 29 19:58:59.341624 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.341604    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e0fe5-59f77594bd-4vwlt"]
Jul 29 19:58:59.350572 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.350547    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"]
Jul 29 19:58:59.354225 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:58:59.354206    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e0fe5-5bdbdb7cb-hn8b8"]
Jul 29 19:59:01.102275 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:01.102235    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32fe5371-0042-4ac5-a6ea-08018ce7147c" path="/var/lib/kubelet/pods/32fe5371-0042-4ac5-a6ea-08018ce7147c/volumes"
Jul 29 19:59:01.102800 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:01.102780    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a76e4ddf-9575-4985-b154-ecf82e514d57" path="/var/lib/kubelet/pods/a76e4ddf-9575-4985-b154-ecf82e514d57/volumes"
Jul 29 19:59:05.968692 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.968649    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"]
Jul 29 19:59:05.969209 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969150    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a76e4ddf-9575-4985-b154-ecf82e514d57" containerName="payload-processing"
Jul 29 19:59:05.969209 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969171    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a76e4ddf-9575-4985-b154-ecf82e514d57" containerName="payload-processing"
Jul 29 19:59:05.969334 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969216    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy"
Jul 29 19:59:05.969334 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969227    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy"
Jul 29 19:59:05.969334 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969271    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32fe5371-0042-4ac5-a6ea-08018ce7147c" containerName="payload-pre-processing"
Jul 29 19:59:05.969334 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969280    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="32fe5371-0042-4ac5-a6ea-08018ce7147c" containerName="payload-pre-processing"
Jul 29 19:59:05.969334 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969290    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy"
Jul 29 19:59:05.969334 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969296    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy"
Jul 29 19:59:05.969659 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969385    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a76e4ddf-9575-4985-b154-ecf82e514d57" containerName="payload-processing"
Jul 29 19:59:05.969659 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969402    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="32fe5371-0042-4ac5-a6ea-08018ce7147c" containerName="payload-pre-processing"
Jul 29 19:59:05.969659 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969413    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy"
Jul 29 19:59:05.969659 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.969424    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="592e40c3-8f7a-4007-87c4-62ab5b0d48fd" containerName="istio-proxy"
Jul 29 19:59:05.973542 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.973526    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:05.976248 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.976228    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 19:59:05.976361 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.976288    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a60880-openshift-default-dockercfg-hsc4k\""
Jul 29 19:59:05.983891 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:05.983868    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"]
Jul 29 19:59:06.027930 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.027895    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-token\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.028082 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.027950    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-podinfo\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.028082 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.027992    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-certs\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.028082 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.028017    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-credential-socket\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.028082 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.028047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istiod-ca-cert\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.028230 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.028096    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-envoy\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.028230 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.028155    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-socket\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.028230 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.028177    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98l46\" (UniqueName: \"kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-kube-api-access-98l46\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.028329 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.028255    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-data\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.129356 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.129324    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istiod-ca-cert\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.129567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.129372    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-envoy\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.129567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.129420    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-socket\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.129567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.129475    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98l46\" (UniqueName: \"kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-kube-api-access-98l46\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.129567 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.129549    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-data\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.129800 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.129586    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-token\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.129800 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.129638    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-podinfo\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.129800 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.129671    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-certs\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.129800 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.129696    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-credential-socket\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.130363 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.130117    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-credential-socket\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.130677 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.130647    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-certs\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.130803 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.130691    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-socket\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.130875 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.130859    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-data\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.130932 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.130869    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istiod-ca-cert\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.132636 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.132616    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-envoy\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.132636 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.132633    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-podinfo\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.137438 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.137419    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-token\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.137626 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.137609    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98l46\" (UniqueName: \"kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-kube-api-access-98l46\") pod \"e2e-shared-a-a60880-openshift-default-6bf448584-khph7\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") " pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.285078 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.284980    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:06.424126 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.424096    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"]
Jul 29 19:59:06.426313 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:59:06.426286    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b331f4_313d_4c17_b3cc_d4b66ab01bc6.slice/crio-0b5bc69440d9c7fb9b9018f4c59be555560e7674bc483aa5dd31c6715bad76a8 WatchSource:0}: Error finding container 0b5bc69440d9c7fb9b9018f4c59be555560e7674bc483aa5dd31c6715bad76a8: Status 404 returned error can't find the container with id 0b5bc69440d9c7fb9b9018f4c59be555560e7674bc483aa5dd31c6715bad76a8
Jul 29 19:59:06.428363 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.428330    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:59:06.428483 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.428406    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:59:06.428483 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:06.428466    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:59:07.374081 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:07.374046    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" event={"ID":"91b331f4-313d-4c17-b3cc-d4b66ab01bc6","Type":"ContainerStarted","Data":"118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027"}
Jul 29 19:59:07.374470 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:07.374100    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" event={"ID":"91b331f4-313d-4c17-b3cc-d4b66ab01bc6","Type":"ContainerStarted","Data":"0b5bc69440d9c7fb9b9018f4c59be555560e7674bc483aa5dd31c6715bad76a8"}
Jul 29 19:59:07.399000 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:07.398954    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" podStartSLOduration=2.398937616 podStartE2EDuration="2.398937616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:59:07.39718936 +0000 UTC m=+2512.892950713" watchObservedRunningTime="2026-07-29 19:59:07.398937616 +0000 UTC m=+2512.894698959"
Jul 29 19:59:08.285570 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:08.285535    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:08.290301 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:08.290275    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:08.378229 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:08.378200    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:08.379123 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:08.379101    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:21.450679 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:21.450647    2566 generic.go:356] "Generic (PLEG): container finished" podID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerID="118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027" exitCode=0
Jul 29 19:59:21.451073 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:21.450722    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" event={"ID":"91b331f4-313d-4c17-b3cc-d4b66ab01bc6","Type":"ContainerDied","Data":"118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027"}
Jul 29 19:59:21.451115 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:21.451106    2566 scope.go:117] "RemoveContainer" containerID="118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027"
Jul 29 19:59:21.452130 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:21.452106    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:59:21.452193 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:21.452180    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:59:21.452249 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:21.452238    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 19:59:22.460024 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:22.459986    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" event={"ID":"91b331f4-313d-4c17-b3cc-d4b66ab01bc6","Type":"ContainerStarted","Data":"dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30"}
Jul 29 19:59:23.285442 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:23.285405    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:23.286678 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:23.286651    2566 patch_prober.go:28] interesting pod/e2e-shared-a-a60880-openshift-default-6bf448584-khph7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 29 19:59:23.286817 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:23.286718    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 29 19:59:23.378509 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:23.378473    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:24.286401 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:24.286363    2566 patch_prober.go:28] interesting pod/e2e-shared-a-a60880-openshift-default-6bf448584-khph7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 29 19:59:24.286790 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:24.286431    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 29 19:59:25.286391 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:25.286351    2566 patch_prober.go:28] interesting pod/e2e-shared-a-a60880-openshift-default-6bf448584-khph7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 29 19:59:25.286600 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:25.286424    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 29 19:59:26.285793 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:26.285759    2566 patch_prober.go:28] interesting pod/e2e-shared-a-a60880-openshift-default-6bf448584-khph7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 29 19:59:26.285980 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:26.285811    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 29 19:59:27.289856 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:27.289826    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:27.290825 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:27.290804    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 19:59:38.595417 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.595380    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"]
Jul 29 19:59:38.598589 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.598569    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 19:59:38.601272 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.601251    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-601aa9\""
Jul 29 19:59:38.601394 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.601356    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-601aa9-dockercfg-rr8bw\""
Jul 29 19:59:38.609605 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.609587    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"]
Jul 29 19:59:38.749039 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.748997    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb\" (UID: \"1a8c94cf-e6c7-4366-b180-c3db7b37b16e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 19:59:38.749218 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.749174    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w82l\" (UniqueName: \"kubernetes.io/projected/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-kube-api-access-5w82l\") pod \"payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb\" (UID: \"1a8c94cf-e6c7-4366-b180-c3db7b37b16e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 19:59:38.850185 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.850105    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w82l\" (UniqueName: \"kubernetes.io/projected/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-kube-api-access-5w82l\") pod \"payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb\" (UID: \"1a8c94cf-e6c7-4366-b180-c3db7b37b16e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 19:59:38.850185 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.850171    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb\" (UID: \"1a8c94cf-e6c7-4366-b180-c3db7b37b16e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 19:59:38.850845 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.850827    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb\" (UID: \"1a8c94cf-e6c7-4366-b180-c3db7b37b16e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 19:59:38.859008 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.858979    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w82l\" (UniqueName: \"kubernetes.io/projected/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-kube-api-access-5w82l\") pod \"payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb\" (UID: \"1a8c94cf-e6c7-4366-b180-c3db7b37b16e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 19:59:38.909832 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:38.909809    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 19:59:39.046011 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:39.045984    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"]
Jul 29 19:59:39.048980 ip-10-0-129-107 kubenswrapper[2566]: W0729 19:59:39.048951    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a8c94cf_e6c7_4366_b180_c3db7b37b16e.slice/crio-6436ba9ef3f429a289b4a1e3c0ee68b77eb9c21458cadc26ed84a67265b5b8ed WatchSource:0}: Error finding container 6436ba9ef3f429a289b4a1e3c0ee68b77eb9c21458cadc26ed84a67265b5b8ed: Status 404 returned error can't find the container with id 6436ba9ef3f429a289b4a1e3c0ee68b77eb9c21458cadc26ed84a67265b5b8ed
Jul 29 19:59:39.545670 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:39.545629    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb" event={"ID":"1a8c94cf-e6c7-4366-b180-c3db7b37b16e","Type":"ContainerStarted","Data":"bb8ee236a76fe6671e5ee30b44754b7ede3da08b3e21e5e2fedcbfe0d8305ad0"}
Jul 29 19:59:39.545857 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:39.545803    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 19:59:39.545857 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:39.545851    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb" event={"ID":"1a8c94cf-e6c7-4366-b180-c3db7b37b16e","Type":"ContainerStarted","Data":"6436ba9ef3f429a289b4a1e3c0ee68b77eb9c21458cadc26ed84a67265b5b8ed"}
Jul 29 19:59:39.564227 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:39.564181    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb" podStartSLOduration=1.564166479 podStartE2EDuration="1.564166479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:59:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:59:39.56248884 +0000 UTC m=+2545.058250208" watchObservedRunningTime="2026-07-29 19:59:39.564166479 +0000 UTC m=+2545.059927820"
Jul 29 19:59:50.554011 ip-10-0-129-107 kubenswrapper[2566]: I0729 19:59:50.553972    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 20:00:18.461759 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.461728    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"]
Jul 29 20:00:18.465382 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.465363    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.469261 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.469239    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"auth-test-model-a60880-kserve-self-signed-certs\""
Jul 29 20:00:18.469373 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.469282    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"kube-root-ca.crt\""
Jul 29 20:00:18.469373 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.469284    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"default-dockercfg-2x8xs\""
Jul 29 20:00:18.469525 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.469239    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"openshift-service-ca.crt\""
Jul 29 20:00:18.475362 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.475341    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"]
Jul 29 20:00:18.522556 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.522525    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-home\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.522716 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.522583    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/122cd532-b81c-42a6-8875-752540c4514a-tls-certs\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.522716 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.522666    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-tmp-dir\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.522839 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.522735    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-dshm\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.522839 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.522772    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-model-cache\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.522839 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.522829    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8smng\" (UniqueName: \"kubernetes.io/projected/122cd532-b81c-42a6-8875-752540c4514a-kube-api-access-8smng\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.623587 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.623553    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-home\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.623766 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.623606    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/122cd532-b81c-42a6-8875-752540c4514a-tls-certs\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.623766 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.623635    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-tmp-dir\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.623766 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.623659    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-dshm\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.623766 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.623677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-model-cache\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.623766 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.623722    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8smng\" (UniqueName: \"kubernetes.io/projected/122cd532-b81c-42a6-8875-752540c4514a-kube-api-access-8smng\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.624034 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.624014    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-home\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.624100 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.624051    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-tmp-dir\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.624100 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.624074    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-model-cache\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.625978 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.625960    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-dshm\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.626135 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.626120    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/122cd532-b81c-42a6-8875-752540c4514a-tls-certs\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.631665 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.631648    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8smng\" (UniqueName: \"kubernetes.io/projected/122cd532-b81c-42a6-8875-752540c4514a-kube-api-access-8smng\") pod \"auth-test-model-a60880-kserve-7f6db97f9f-gj886\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") " pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.776520 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.776417    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:18.913975 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:18.913952    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"]
Jul 29 20:00:18.915501 ip-10-0-129-107 kubenswrapper[2566]: W0729 20:00:18.915477    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod122cd532_b81c_42a6_8875_752540c4514a.slice/crio-2f2e4c1e64f9dfa2f1ed7c04000f41a91009c8293bda1382f08e5c1823ecfb7d WatchSource:0}: Error finding container 2f2e4c1e64f9dfa2f1ed7c04000f41a91009c8293bda1382f08e5c1823ecfb7d: Status 404 returned error can't find the container with id 2f2e4c1e64f9dfa2f1ed7c04000f41a91009c8293bda1382f08e5c1823ecfb7d
Jul 29 20:00:19.748783 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:19.748737    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886" event={"ID":"122cd532-b81c-42a6-8875-752540c4514a","Type":"ContainerStarted","Data":"8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc"}
Jul 29 20:00:19.749148 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:19.748790    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886" event={"ID":"122cd532-b81c-42a6-8875-752540c4514a","Type":"ContainerStarted","Data":"2f2e4c1e64f9dfa2f1ed7c04000f41a91009c8293bda1382f08e5c1823ecfb7d"}
Jul 29 20:00:19.770556 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:19.770511    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886" podStartSLOduration=1.7704971189999998 podStartE2EDuration="1.770497119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:00:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:00:19.770177117 +0000 UTC m=+2585.265938460" watchObservedRunningTime="2026-07-29 20:00:19.770497119 +0000 UTC m=+2585.266258481"
Jul 29 20:00:28.777227 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:28.777194    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:28.777716 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:28.777348    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:28.791759 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:28.791734    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:00:29.806509 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:00:29.806481    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:05:19.498704 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:19.498625    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"]
Jul 29 20:05:34.518142 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.518098    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886" podUID="122cd532-b81c-42a6-8875-752540c4514a" containerName="main" containerID="cri-o://8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc" gracePeriod=45
Jul 29 20:05:34.765100 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.765075    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:05:34.886853 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.886764    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-tmp-dir\") pod \"122cd532-b81c-42a6-8875-752540c4514a\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") "
Jul 29 20:05:34.886853 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.886802    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-dshm\") pod \"122cd532-b81c-42a6-8875-752540c4514a\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") "
Jul 29 20:05:34.887087 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.886854    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/122cd532-b81c-42a6-8875-752540c4514a-tls-certs\") pod \"122cd532-b81c-42a6-8875-752540c4514a\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") "
Jul 29 20:05:34.887087 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.886906    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-home\") pod \"122cd532-b81c-42a6-8875-752540c4514a\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") "
Jul 29 20:05:34.887087 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.886983    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-model-cache\") pod \"122cd532-b81c-42a6-8875-752540c4514a\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") "
Jul 29 20:05:34.887087 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.887021    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8smng\" (UniqueName: \"kubernetes.io/projected/122cd532-b81c-42a6-8875-752540c4514a-kube-api-access-8smng\") pod \"122cd532-b81c-42a6-8875-752540c4514a\" (UID: \"122cd532-b81c-42a6-8875-752540c4514a\") "
Jul 29 20:05:34.887087 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.887055    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "122cd532-b81c-42a6-8875-752540c4514a" (UID: "122cd532-b81c-42a6-8875-752540c4514a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:05:34.887357 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.887211    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-home" (OuterVolumeSpecName: "home") pod "122cd532-b81c-42a6-8875-752540c4514a" (UID: "122cd532-b81c-42a6-8875-752540c4514a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:05:34.887357 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.887285    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-model-cache" (OuterVolumeSpecName: "model-cache") pod "122cd532-b81c-42a6-8875-752540c4514a" (UID: "122cd532-b81c-42a6-8875-752540c4514a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:05:34.887357 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.887334    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-home\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:05:34.887552 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.887362    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-tmp-dir\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:05:34.889136 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.889109    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-dshm" (OuterVolumeSpecName: "dshm") pod "122cd532-b81c-42a6-8875-752540c4514a" (UID: "122cd532-b81c-42a6-8875-752540c4514a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:05:34.889255 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.889117    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/122cd532-b81c-42a6-8875-752540c4514a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "122cd532-b81c-42a6-8875-752540c4514a" (UID: "122cd532-b81c-42a6-8875-752540c4514a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:05:34.889255 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.889190    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/122cd532-b81c-42a6-8875-752540c4514a-kube-api-access-8smng" (OuterVolumeSpecName: "kube-api-access-8smng") pod "122cd532-b81c-42a6-8875-752540c4514a" (UID: "122cd532-b81c-42a6-8875-752540c4514a"). InnerVolumeSpecName "kube-api-access-8smng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:05:34.987861 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.987821    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-dshm\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:05:34.987861 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.987855    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/122cd532-b81c-42a6-8875-752540c4514a-tls-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:05:34.987861 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.987868    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/122cd532-b81c-42a6-8875-752540c4514a-model-cache\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:05:34.988092 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:34.987880    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8smng\" (UniqueName: \"kubernetes.io/projected/122cd532-b81c-42a6-8875-752540c4514a-kube-api-access-8smng\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:05:35.291335 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.291298    2566 generic.go:356] "Generic (PLEG): container finished" podID="122cd532-b81c-42a6-8875-752540c4514a" containerID="8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc" exitCode=0
Jul 29 20:05:35.291529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.291353    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"
Jul 29 20:05:35.291529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.291380    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886" event={"ID":"122cd532-b81c-42a6-8875-752540c4514a","Type":"ContainerDied","Data":"8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc"}
Jul 29 20:05:35.291529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.291424    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886" event={"ID":"122cd532-b81c-42a6-8875-752540c4514a","Type":"ContainerDied","Data":"2f2e4c1e64f9dfa2f1ed7c04000f41a91009c8293bda1382f08e5c1823ecfb7d"}
Jul 29 20:05:35.291529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.291442    2566 scope.go:117] "RemoveContainer" containerID="8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc"
Jul 29 20:05:35.300345 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.300327    2566 scope.go:117] "RemoveContainer" containerID="8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc"
Jul 29 20:05:35.300625 ip-10-0-129-107 kubenswrapper[2566]: E0729 20:05:35.300607    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc\": container with ID starting with 8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc not found: ID does not exist" containerID="8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc"
Jul 29 20:05:35.300708 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.300637    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc"} err="failed to get container status \"8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc\": rpc error: code = NotFound desc = could not find container \"8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc\": container with ID starting with 8481c210bd535e043ab1c6cda2374394b4e645f7c12bbdd8a87d07be508400bc not found: ID does not exist"
Jul 29 20:05:35.309144 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.309119    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"]
Jul 29 20:05:35.312781 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.312762    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/auth-test-model-a60880-kserve-7f6db97f9f-gj886"]
Jul 29 20:05:35.579742 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.579671    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"]
Jul 29 20:05:35.580077 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.580067    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="122cd532-b81c-42a6-8875-752540c4514a" containerName="main"
Jul 29 20:05:35.580119 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.580078    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="122cd532-b81c-42a6-8875-752540c4514a" containerName="main"
Jul 29 20:05:35.580166 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.580156    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="122cd532-b81c-42a6-8875-752540c4514a" containerName="main"
Jul 29 20:05:35.584614 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.584592    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.587678 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.587655    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"kube-root-ca.crt\""
Jul 29 20:05:35.588175 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.588152    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"sub-test-model-a60880-kserve-self-signed-certs\""
Jul 29 20:05:35.588643 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.588430    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"openshift-service-ca.crt\""
Jul 29 20:05:35.588643 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.588480    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"default-dockercfg-2x8xs\""
Jul 29 20:05:35.594101 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.594078    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"]
Jul 29 20:05:35.694248 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.694222    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3312531-c2f1-4e33-8f32-b075b98b4707-tls-certs\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.694414 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.694279    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-home\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.694414 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.694296    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcjrb\" (UniqueName: \"kubernetes.io/projected/f3312531-c2f1-4e33-8f32-b075b98b4707-kube-api-access-pcjrb\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.694414 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.694382    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-tmp-dir\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.694614 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.694431    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-dshm\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.694614 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.694520    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-model-cache\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.795243 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.795211    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-home\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.795243 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.795242    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcjrb\" (UniqueName: \"kubernetes.io/projected/f3312531-c2f1-4e33-8f32-b075b98b4707-kube-api-access-pcjrb\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.795479 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.795269    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-tmp-dir\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.795479 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.795291    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-dshm\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.795479 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.795316    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-model-cache\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.795479 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.795374    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3312531-c2f1-4e33-8f32-b075b98b4707-tls-certs\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.795687 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.795668    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-home\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.795752 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.795736    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-tmp-dir\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.795877 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.795857    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-model-cache\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.797664 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.797638    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-dshm\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.797834 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.797819    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3312531-c2f1-4e33-8f32-b075b98b4707-tls-certs\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.804319 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.804291    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcjrb\" (UniqueName: \"kubernetes.io/projected/f3312531-c2f1-4e33-8f32-b075b98b4707-kube-api-access-pcjrb\") pod \"sub-test-model-a60880-kserve-7b979c75c7-fxmww\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") " pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:35.896306 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:35.896237    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:36.030747 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:36.030717    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"]
Jul 29 20:05:36.032557 ip-10-0-129-107 kubenswrapper[2566]: W0729 20:05:36.032522    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3312531_c2f1_4e33_8f32_b075b98b4707.slice/crio-639d6e5cde978977b2a0ede7ab63be86786806faf57b44da99faadde1176a4a5 WatchSource:0}: Error finding container 639d6e5cde978977b2a0ede7ab63be86786806faf57b44da99faadde1176a4a5: Status 404 returned error can't find the container with id 639d6e5cde978977b2a0ede7ab63be86786806faf57b44da99faadde1176a4a5
Jul 29 20:05:36.299211 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:36.299173    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww" event={"ID":"f3312531-c2f1-4e33-8f32-b075b98b4707","Type":"ContainerStarted","Data":"dc9b379977e20fafda1f5a11069b73962f495eb1aa21bc498edef8dbacaa4386"}
Jul 29 20:05:36.299359 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:36.299218    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww" event={"ID":"f3312531-c2f1-4e33-8f32-b075b98b4707","Type":"ContainerStarted","Data":"639d6e5cde978977b2a0ede7ab63be86786806faf57b44da99faadde1176a4a5"}
Jul 29 20:05:36.318310 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:36.318262    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww" podStartSLOduration=1.3182477590000001 podStartE2EDuration="1.318247759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:05:36.317873194 +0000 UTC m=+2901.813634537" watchObservedRunningTime="2026-07-29 20:05:36.318247759 +0000 UTC m=+2901.814009100"
Jul 29 20:05:37.101563 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:37.101529    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="122cd532-b81c-42a6-8875-752540c4514a" path="/var/lib/kubelet/pods/122cd532-b81c-42a6-8875-752540c4514a/volumes"
Jul 29 20:05:45.896775 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:45.896740    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:45.897246 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:45.896791    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:45.909143 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:45.909123    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:05:46.360807 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:05:46.360776    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:07:19.642346 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:19.642311    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"]
Jul 29 20:07:34.661610 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:34.661536    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww" podUID="f3312531-c2f1-4e33-8f32-b075b98b4707" containerName="main" containerID="cri-o://dc9b379977e20fafda1f5a11069b73962f495eb1aa21bc498edef8dbacaa4386" gracePeriod=45
Jul 29 20:07:34.880182 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:34.880152    2566 generic.go:356] "Generic (PLEG): container finished" podID="f3312531-c2f1-4e33-8f32-b075b98b4707" containerID="dc9b379977e20fafda1f5a11069b73962f495eb1aa21bc498edef8dbacaa4386" exitCode=0
Jul 29 20:07:34.880371 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:34.880209    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww" event={"ID":"f3312531-c2f1-4e33-8f32-b075b98b4707","Type":"ContainerDied","Data":"dc9b379977e20fafda1f5a11069b73962f495eb1aa21bc498edef8dbacaa4386"}
Jul 29 20:07:34.916364 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:34.916297    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:07:35.076007 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.075970    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-dshm\") pod \"f3312531-c2f1-4e33-8f32-b075b98b4707\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") "
Jul 29 20:07:35.076007 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.076009    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-home\") pod \"f3312531-c2f1-4e33-8f32-b075b98b4707\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") "
Jul 29 20:07:35.076221 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.076033    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-tmp-dir\") pod \"f3312531-c2f1-4e33-8f32-b075b98b4707\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") "
Jul 29 20:07:35.076221 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.076075    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3312531-c2f1-4e33-8f32-b075b98b4707-tls-certs\") pod \"f3312531-c2f1-4e33-8f32-b075b98b4707\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") "
Jul 29 20:07:35.076221 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.076114    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-model-cache\") pod \"f3312531-c2f1-4e33-8f32-b075b98b4707\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") "
Jul 29 20:07:35.076385 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.076319    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcjrb\" (UniqueName: \"kubernetes.io/projected/f3312531-c2f1-4e33-8f32-b075b98b4707-kube-api-access-pcjrb\") pod \"f3312531-c2f1-4e33-8f32-b075b98b4707\" (UID: \"f3312531-c2f1-4e33-8f32-b075b98b4707\") "
Jul 29 20:07:35.076436 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.076318    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-home" (OuterVolumeSpecName: "home") pod "f3312531-c2f1-4e33-8f32-b075b98b4707" (UID: "f3312531-c2f1-4e33-8f32-b075b98b4707"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:07:35.076436 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.076415    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3312531-c2f1-4e33-8f32-b075b98b4707" (UID: "f3312531-c2f1-4e33-8f32-b075b98b4707"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:07:35.076582 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.076327    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3312531-c2f1-4e33-8f32-b075b98b4707" (UID: "f3312531-c2f1-4e33-8f32-b075b98b4707"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:07:35.076739 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.076719    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-model-cache\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:07:35.076801 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.076744    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-home\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:07:35.076801 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.076757    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-tmp-dir\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:07:35.078375 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.078228    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3312531-c2f1-4e33-8f32-b075b98b4707-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3312531-c2f1-4e33-8f32-b075b98b4707" (UID: "f3312531-c2f1-4e33-8f32-b075b98b4707"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:07:35.078375 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.078302    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-dshm" (OuterVolumeSpecName: "dshm") pod "f3312531-c2f1-4e33-8f32-b075b98b4707" (UID: "f3312531-c2f1-4e33-8f32-b075b98b4707"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:07:35.078375 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.078304    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3312531-c2f1-4e33-8f32-b075b98b4707-kube-api-access-pcjrb" (OuterVolumeSpecName: "kube-api-access-pcjrb") pod "f3312531-c2f1-4e33-8f32-b075b98b4707" (UID: "f3312531-c2f1-4e33-8f32-b075b98b4707"). InnerVolumeSpecName "kube-api-access-pcjrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:07:35.178315 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.178280    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3312531-c2f1-4e33-8f32-b075b98b4707-tls-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:07:35.178315 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.178310    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcjrb\" (UniqueName: \"kubernetes.io/projected/f3312531-c2f1-4e33-8f32-b075b98b4707-kube-api-access-pcjrb\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:07:35.178563 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.178324    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3312531-c2f1-4e33-8f32-b075b98b4707-dshm\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:07:35.886920 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.886890    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"
Jul 29 20:07:35.887366 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.886937    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww" event={"ID":"f3312531-c2f1-4e33-8f32-b075b98b4707","Type":"ContainerDied","Data":"639d6e5cde978977b2a0ede7ab63be86786806faf57b44da99faadde1176a4a5"}
Jul 29 20:07:35.887366 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.886989    2566 scope.go:117] "RemoveContainer" containerID="dc9b379977e20fafda1f5a11069b73962f495eb1aa21bc498edef8dbacaa4386"
Jul 29 20:07:35.906507 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.906477    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"]
Jul 29 20:07:35.910060 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:35.910030    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/sub-test-model-a60880-kserve-7b979c75c7-fxmww"]
Jul 29 20:07:37.101786 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:37.101751    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3312531-c2f1-4e33-8f32-b075b98b4707" path="/var/lib/kubelet/pods/f3312531-c2f1-4e33-8f32-b075b98b4707/volumes"
Jul 29 20:07:40.150569 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.150529    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"]
Jul 29 20:07:40.150981 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.150969    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3312531-c2f1-4e33-8f32-b075b98b4707" containerName="main"
Jul 29 20:07:40.151025 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.150984    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3312531-c2f1-4e33-8f32-b075b98b4707" containerName="main"
Jul 29 20:07:40.151068 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.151041    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3312531-c2f1-4e33-8f32-b075b98b4707" containerName="main"
Jul 29 20:07:40.155433 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.155415    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.158258 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.158236    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"kube-root-ca.crt\""
Jul 29 20:07:40.159509 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.159488    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"rate-test-model-a60880-kserve-self-signed-certs\""
Jul 29 20:07:40.159585 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.159489    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"openshift-service-ca.crt\""
Jul 29 20:07:40.159585 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.159529    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a60880\"/\"default-dockercfg-2x8xs\""
Jul 29 20:07:40.164565 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.164539    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"]
Jul 29 20:07:40.226056 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.226020    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-dshm\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.226238 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.226103    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/657477bb-f35c-4160-8d81-d6a968992520-tls-certs\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.226238 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.226184    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g4pt\" (UniqueName: \"kubernetes.io/projected/657477bb-f35c-4160-8d81-d6a968992520-kube-api-access-8g4pt\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.226361 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.226244    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-home\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.226361 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.226295    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-tmp-dir\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.226361 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.226344    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-model-cache\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.327913 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.327877    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-dshm\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.327913 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.327921    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/657477bb-f35c-4160-8d81-d6a968992520-tls-certs\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.328173 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.327956    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g4pt\" (UniqueName: \"kubernetes.io/projected/657477bb-f35c-4160-8d81-d6a968992520-kube-api-access-8g4pt\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.328173 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.327976    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-home\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.328173 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.328001    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-tmp-dir\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.328173 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.328030    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-model-cache\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.328467 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.328371    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-tmp-dir\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.328467 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.328427    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-home\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.328561 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.328483    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-model-cache\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.330268 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.330248    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-dshm\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.330415 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.330394    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/657477bb-f35c-4160-8d81-d6a968992520-tls-certs\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.336995 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.336969    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g4pt\" (UniqueName: \"kubernetes.io/projected/657477bb-f35c-4160-8d81-d6a968992520-kube-api-access-8g4pt\") pod \"rate-test-model-a60880-kserve-5d8c6c66c-g8kvp\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") " pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.465901 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.465865    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:40.601124 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.601096    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"]
Jul 29 20:07:40.603487 ip-10-0-129-107 kubenswrapper[2566]: W0729 20:07:40.603444    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod657477bb_f35c_4160_8d81_d6a968992520.slice/crio-10ea8871564406355e4190e8d18d30d666c35d1374edbc5fe180e533a73ebe3f WatchSource:0}: Error finding container 10ea8871564406355e4190e8d18d30d666c35d1374edbc5fe180e533a73ebe3f: Status 404 returned error can't find the container with id 10ea8871564406355e4190e8d18d30d666c35d1374edbc5fe180e533a73ebe3f
Jul 29 20:07:40.913895 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.913808    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp" event={"ID":"657477bb-f35c-4160-8d81-d6a968992520","Type":"ContainerStarted","Data":"d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112"}
Jul 29 20:07:40.913895 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.913849    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp" event={"ID":"657477bb-f35c-4160-8d81-d6a968992520","Type":"ContainerStarted","Data":"10ea8871564406355e4190e8d18d30d666c35d1374edbc5fe180e533a73ebe3f"}
Jul 29 20:07:40.932328 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:40.932276    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp" podStartSLOduration=0.932261256 podStartE2EDuration="932.261256ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:07:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:07:40.931316872 +0000 UTC m=+3026.427078214" watchObservedRunningTime="2026-07-29 20:07:40.932261256 +0000 UTC m=+3026.428022598"
Jul 29 20:07:50.467006 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:50.466963    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:50.467541 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:50.467026    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:50.479709 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:50.479682    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:07:50.974302 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:07:50.974274    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:09:16.790557 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:16.790522    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"]
Jul 29 20:09:31.809216 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:31.809158    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp" podUID="657477bb-f35c-4160-8d81-d6a968992520" containerName="main" containerID="cri-o://d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112" gracePeriod=45
Jul 29 20:09:32.060894 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.060826    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:09:32.141865 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.141818    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-home\") pod \"657477bb-f35c-4160-8d81-d6a968992520\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") "
Jul 29 20:09:32.142062 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.141897    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8g4pt\" (UniqueName: \"kubernetes.io/projected/657477bb-f35c-4160-8d81-d6a968992520-kube-api-access-8g4pt\") pod \"657477bb-f35c-4160-8d81-d6a968992520\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") "
Jul 29 20:09:32.142062 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.141942    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/657477bb-f35c-4160-8d81-d6a968992520-tls-certs\") pod \"657477bb-f35c-4160-8d81-d6a968992520\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") "
Jul 29 20:09:32.142062 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.141966    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-tmp-dir\") pod \"657477bb-f35c-4160-8d81-d6a968992520\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") "
Jul 29 20:09:32.142240 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.142100    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-dshm\") pod \"657477bb-f35c-4160-8d81-d6a968992520\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") "
Jul 29 20:09:32.142240 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.142126    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-home" (OuterVolumeSpecName: "home") pod "657477bb-f35c-4160-8d81-d6a968992520" (UID: "657477bb-f35c-4160-8d81-d6a968992520"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:09:32.142240 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.142158    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-model-cache\") pod \"657477bb-f35c-4160-8d81-d6a968992520\" (UID: \"657477bb-f35c-4160-8d81-d6a968992520\") "
Jul 29 20:09:32.142240 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.142220    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "657477bb-f35c-4160-8d81-d6a968992520" (UID: "657477bb-f35c-4160-8d81-d6a968992520"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:09:32.142520 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.142498    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-model-cache" (OuterVolumeSpecName: "model-cache") pod "657477bb-f35c-4160-8d81-d6a968992520" (UID: "657477bb-f35c-4160-8d81-d6a968992520"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:09:32.142623 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.142607    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-tmp-dir\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:09:32.142679 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.142627    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-model-cache\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:09:32.144187 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.144148    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/657477bb-f35c-4160-8d81-d6a968992520-kube-api-access-8g4pt" (OuterVolumeSpecName: "kube-api-access-8g4pt") pod "657477bb-f35c-4160-8d81-d6a968992520" (UID: "657477bb-f35c-4160-8d81-d6a968992520"). InnerVolumeSpecName "kube-api-access-8g4pt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:09:32.144321 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.144206    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/657477bb-f35c-4160-8d81-d6a968992520-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "657477bb-f35c-4160-8d81-d6a968992520" (UID: "657477bb-f35c-4160-8d81-d6a968992520"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:09:32.144816 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.144794    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-dshm" (OuterVolumeSpecName: "dshm") pod "657477bb-f35c-4160-8d81-d6a968992520" (UID: "657477bb-f35c-4160-8d81-d6a968992520"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:09:32.243560 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.243521    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-home\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:09:32.243560 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.243555    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g4pt\" (UniqueName: \"kubernetes.io/projected/657477bb-f35c-4160-8d81-d6a968992520-kube-api-access-8g4pt\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:09:32.243560 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.243566    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/657477bb-f35c-4160-8d81-d6a968992520-tls-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:09:32.243806 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.243576    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/657477bb-f35c-4160-8d81-d6a968992520-dshm\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:09:32.498211 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.498170    2566 generic.go:356] "Generic (PLEG): container finished" podID="657477bb-f35c-4160-8d81-d6a968992520" containerID="d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112" exitCode=0
Jul 29 20:09:32.498211 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.498215    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"
Jul 29 20:09:32.498461 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.498260    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp" event={"ID":"657477bb-f35c-4160-8d81-d6a968992520","Type":"ContainerDied","Data":"d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112"}
Jul 29 20:09:32.498461 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.498307    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp" event={"ID":"657477bb-f35c-4160-8d81-d6a968992520","Type":"ContainerDied","Data":"10ea8871564406355e4190e8d18d30d666c35d1374edbc5fe180e533a73ebe3f"}
Jul 29 20:09:32.498461 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.498325    2566 scope.go:117] "RemoveContainer" containerID="d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112"
Jul 29 20:09:32.508252 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.508234    2566 scope.go:117] "RemoveContainer" containerID="d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112"
Jul 29 20:09:32.508555 ip-10-0-129-107 kubenswrapper[2566]: E0729 20:09:32.508535    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112\": container with ID starting with d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112 not found: ID does not exist" containerID="d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112"
Jul 29 20:09:32.508618 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.508564    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112"} err="failed to get container status \"d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112\": rpc error: code = NotFound desc = could not find container \"d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112\": container with ID starting with d06505d3dc5273f2a7bf772bdc054e876f2e4662e1e881c09ea0916a23530112 not found: ID does not exist"
Jul 29 20:09:32.520031 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.520008    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"]
Jul 29 20:09:32.523678 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:32.523655    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a60880/rate-test-model-a60880-kserve-5d8c6c66c-g8kvp"]
Jul 29 20:09:33.102773 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:33.102737    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="657477bb-f35c-4160-8d81-d6a968992520" path="/var/lib/kubelet/pods/657477bb-f35c-4160-8d81-d6a968992520/volumes"
Jul 29 20:09:34.477570 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.477536    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7535-140196111117312"]
Jul 29 20:09:34.478014 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.477999    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="657477bb-f35c-4160-8d81-d6a968992520" containerName="main"
Jul 29 20:09:34.478053 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.478015    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="657477bb-f35c-4160-8d81-d6a968992520" containerName="main"
Jul 29 20:09:34.478099 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.478089    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="657477bb-f35c-4160-8d81-d6a968992520" containerName="main"
Jul 29 20:09:34.482836 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.482815    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140196111117312"
Jul 29 20:09:34.485237 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.485203    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7535-140196111117312"]
Jul 29 20:09:34.485469 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.485438    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lq9lk\""
Jul 29 20:09:34.566502 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.566466    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8jwg\" (UniqueName: \"kubernetes.io/projected/9804959c-c460-412c-8ce5-51cbefac4de9-kube-api-access-b8jwg\") pod \"test-curl-7535-140196111117312\" (UID: \"9804959c-c460-412c-8ce5-51cbefac4de9\") " pod="opendatahub/test-curl-7535-140196111117312"
Jul 29 20:09:34.667270 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.667237    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8jwg\" (UniqueName: \"kubernetes.io/projected/9804959c-c460-412c-8ce5-51cbefac4de9-kube-api-access-b8jwg\") pod \"test-curl-7535-140196111117312\" (UID: \"9804959c-c460-412c-8ce5-51cbefac4de9\") " pod="opendatahub/test-curl-7535-140196111117312"
Jul 29 20:09:34.675412 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.675379    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8jwg\" (UniqueName: \"kubernetes.io/projected/9804959c-c460-412c-8ce5-51cbefac4de9-kube-api-access-b8jwg\") pod \"test-curl-7535-140196111117312\" (UID: \"9804959c-c460-412c-8ce5-51cbefac4de9\") " pod="opendatahub/test-curl-7535-140196111117312"
Jul 29 20:09:34.792786 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.792711    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140196111117312"
Jul 29 20:09:34.927177 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.927145    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7535-140196111117312"]
Jul 29 20:09:34.930187 ip-10-0-129-107 kubenswrapper[2566]: W0729 20:09:34.930154    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9804959c_c460_412c_8ce5_51cbefac4de9.slice/crio-1d5c5ae51cab8196cbcead9946800a086dcb4f3efda851b657847fb5f4967ebd WatchSource:0}: Error finding container 1d5c5ae51cab8196cbcead9946800a086dcb4f3efda851b657847fb5f4967ebd: Status 404 returned error can't find the container with id 1d5c5ae51cab8196cbcead9946800a086dcb4f3efda851b657847fb5f4967ebd
Jul 29 20:09:34.931590 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:34.931571    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 20:09:35.517135 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:35.517095    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7535-140196111117312" event={"ID":"9804959c-c460-412c-8ce5-51cbefac4de9","Type":"ContainerStarted","Data":"1d5c5ae51cab8196cbcead9946800a086dcb4f3efda851b657847fb5f4967ebd"}
Jul 29 20:09:36.523233 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:36.523147    2566 generic.go:356] "Generic (PLEG): container finished" podID="9804959c-c460-412c-8ce5-51cbefac4de9" containerID="db8e0f501a70e14e89b4103cd94342a6095ccb0ff74cac48f42c895dcab43127" exitCode=0
Jul 29 20:09:36.523233 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:36.523214    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7535-140196111117312" event={"ID":"9804959c-c460-412c-8ce5-51cbefac4de9","Type":"ContainerDied","Data":"db8e0f501a70e14e89b4103cd94342a6095ccb0ff74cac48f42c895dcab43127"}
Jul 29 20:09:37.659258 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:37.659235    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140196111117312"
Jul 29 20:09:37.695075 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:37.695045    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7535-140196111117312_9804959c-c460-412c-8ce5-51cbefac4de9/test-curl-7535-140196111117312/0.log"
Jul 29 20:09:37.719203 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:37.719175    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7535-140196111117312"]
Jul 29 20:09:37.723084 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:37.723060    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7535-140196111117312"]
Jul 29 20:09:37.795232 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:37.795150    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8jwg\" (UniqueName: \"kubernetes.io/projected/9804959c-c460-412c-8ce5-51cbefac4de9-kube-api-access-b8jwg\") pod \"9804959c-c460-412c-8ce5-51cbefac4de9\" (UID: \"9804959c-c460-412c-8ce5-51cbefac4de9\") "
Jul 29 20:09:37.797232 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:37.797209    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9804959c-c460-412c-8ce5-51cbefac4de9-kube-api-access-b8jwg" (OuterVolumeSpecName: "kube-api-access-b8jwg") pod "9804959c-c460-412c-8ce5-51cbefac4de9" (UID: "9804959c-c460-412c-8ce5-51cbefac4de9"). InnerVolumeSpecName "kube-api-access-b8jwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:09:37.896687 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:37.896645    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8jwg\" (UniqueName: \"kubernetes.io/projected/9804959c-c460-412c-8ce5-51cbefac4de9-kube-api-access-b8jwg\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:09:38.535692 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:38.535648    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140196111117312"
Jul 29 20:09:38.535692 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:38.535678    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d5c5ae51cab8196cbcead9946800a086dcb4f3efda851b657847fb5f4967ebd"
Jul 29 20:09:38.548230 ip-10-0-129-107 kubenswrapper[2566]: E0729 20:09:38.548199    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7535-140196111117312\" is forbidden: User \"system:node:ip-10-0-129-107.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-107.ec2.internal' and this object" podUID="9804959c-c460-412c-8ce5-51cbefac4de9" pod="opendatahub/test-curl-7535-140196111117312"
Jul 29 20:09:39.102694 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:39.102662    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9804959c-c460-412c-8ce5-51cbefac4de9" path="/var/lib/kubelet/pods/9804959c-c460-412c-8ce5-51cbefac4de9/volumes"
Jul 29 20:09:41.902045 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:41.902009    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"]
Jul 29 20:09:41.902658 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:41.902619    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9804959c-c460-412c-8ce5-51cbefac4de9" containerName="test-curl-7535-140196111117312"
Jul 29 20:09:41.902658 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:41.902645    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9804959c-c460-412c-8ce5-51cbefac4de9" containerName="test-curl-7535-140196111117312"
Jul 29 20:09:41.902813 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:41.902759    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9804959c-c460-412c-8ce5-51cbefac4de9" containerName="test-curl-7535-140196111117312"
Jul 29 20:09:41.905191 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:41.905168    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:09:41.908420 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:41.908401    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-fb0399-dockercfg-mskq5\""
Jul 29 20:09:41.908565 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:41.908484    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-fb0399\""
Jul 29 20:09:41.915740 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:41.915714    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"]
Jul 29 20:09:41.921659 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:41.921637    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"]
Jul 29 20:09:41.924369 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:41.924353    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:09:41.937591 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:41.937564    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"]
Jul 29 20:09:42.045381 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.045325    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8f2m\" (UniqueName: \"kubernetes.io/projected/51a53f77-9600-44b1-be67-1c556b362c01-kube-api-access-l8f2m\") pod \"payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf\" (UID: \"51a53f77-9600-44b1-be67-1c556b362c01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:09:42.045381 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.045372    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a53f77-9600-44b1-be67-1c556b362c01-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf\" (UID: \"51a53f77-9600-44b1-be67-1c556b362c01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:09:42.045696 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.045409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/833e778b-3ac4-4b31-939b-1694bfd028fe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924\" (UID: \"833e778b-3ac4-4b31-939b-1694bfd028fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:09:42.045696 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.045502    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nvzz\" (UniqueName: \"kubernetes.io/projected/833e778b-3ac4-4b31-939b-1694bfd028fe-kube-api-access-6nvzz\") pod \"payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924\" (UID: \"833e778b-3ac4-4b31-939b-1694bfd028fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:09:42.146410 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.146370    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8f2m\" (UniqueName: \"kubernetes.io/projected/51a53f77-9600-44b1-be67-1c556b362c01-kube-api-access-l8f2m\") pod \"payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf\" (UID: \"51a53f77-9600-44b1-be67-1c556b362c01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:09:42.146410 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.146416    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a53f77-9600-44b1-be67-1c556b362c01-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf\" (UID: \"51a53f77-9600-44b1-be67-1c556b362c01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:09:42.146701 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.146470    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/833e778b-3ac4-4b31-939b-1694bfd028fe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924\" (UID: \"833e778b-3ac4-4b31-939b-1694bfd028fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:09:42.146701 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.146546    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nvzz\" (UniqueName: \"kubernetes.io/projected/833e778b-3ac4-4b31-939b-1694bfd028fe-kube-api-access-6nvzz\") pod \"payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924\" (UID: \"833e778b-3ac4-4b31-939b-1694bfd028fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:09:42.147181 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.147158    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a53f77-9600-44b1-be67-1c556b362c01-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf\" (UID: \"51a53f77-9600-44b1-be67-1c556b362c01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:09:42.147271 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.147252    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/833e778b-3ac4-4b31-939b-1694bfd028fe-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924\" (UID: \"833e778b-3ac4-4b31-939b-1694bfd028fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:09:42.155062 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.154988    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8f2m\" (UniqueName: \"kubernetes.io/projected/51a53f77-9600-44b1-be67-1c556b362c01-kube-api-access-l8f2m\") pod \"payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf\" (UID: \"51a53f77-9600-44b1-be67-1c556b362c01\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:09:42.155186 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.155061    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nvzz\" (UniqueName: \"kubernetes.io/projected/833e778b-3ac4-4b31-939b-1694bfd028fe-kube-api-access-6nvzz\") pod \"payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924\" (UID: \"833e778b-3ac4-4b31-939b-1694bfd028fe\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:09:42.216791 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.216758    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:09:42.235305 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.235279    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:09:42.367176 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.367087    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"]
Jul 29 20:09:42.368767 ip-10-0-129-107 kubenswrapper[2566]: W0729 20:09:42.368726    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a53f77_9600_44b1_be67_1c556b362c01.slice/crio-835060751fab081403ed1e1ace63e67d80f26d8535f48cc3fb5be99672239406 WatchSource:0}: Error finding container 835060751fab081403ed1e1ace63e67d80f26d8535f48cc3fb5be99672239406: Status 404 returned error can't find the container with id 835060751fab081403ed1e1ace63e67d80f26d8535f48cc3fb5be99672239406
Jul 29 20:09:42.400006 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.399985    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"]
Jul 29 20:09:42.401821 ip-10-0-129-107 kubenswrapper[2566]: W0729 20:09:42.401797    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod833e778b_3ac4_4b31_939b_1694bfd028fe.slice/crio-6cd0cdd80e6363c86246ff7594c26ccccd71ad1dddec2ee1561b284096333299 WatchSource:0}: Error finding container 6cd0cdd80e6363c86246ff7594c26ccccd71ad1dddec2ee1561b284096333299: Status 404 returned error can't find the container with id 6cd0cdd80e6363c86246ff7594c26ccccd71ad1dddec2ee1561b284096333299
Jul 29 20:09:42.558351 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.558301    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924" event={"ID":"833e778b-3ac4-4b31-939b-1694bfd028fe","Type":"ContainerStarted","Data":"bf5c1a1e8070fd07d04564bd24391de572c8eeb40e353daddbd1d07b19b73416"}
Jul 29 20:09:42.558581 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.558391    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924" event={"ID":"833e778b-3ac4-4b31-939b-1694bfd028fe","Type":"ContainerStarted","Data":"6cd0cdd80e6363c86246ff7594c26ccccd71ad1dddec2ee1561b284096333299"}
Jul 29 20:09:42.558581 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.558471    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:09:42.560476 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.560418    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf" event={"ID":"51a53f77-9600-44b1-be67-1c556b362c01","Type":"ContainerStarted","Data":"6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877"}
Jul 29 20:09:42.560621 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.560485    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf" event={"ID":"51a53f77-9600-44b1-be67-1c556b362c01","Type":"ContainerStarted","Data":"835060751fab081403ed1e1ace63e67d80f26d8535f48cc3fb5be99672239406"}
Jul 29 20:09:42.560621 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.560530    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:09:42.574378 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.574318    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924" podStartSLOduration=1.574300968 podStartE2EDuration="1.574300968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:09:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:09:42.573965124 +0000 UTC m=+3148.069726492" watchObservedRunningTime="2026-07-29 20:09:42.574300968 +0000 UTC m=+3148.070062311"
Jul 29 20:09:42.590010 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:42.589961    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf" podStartSLOduration=1.5899460539999999 podStartE2EDuration="1.589946054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:09:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:09:42.589040917 +0000 UTC m=+3148.084802276" watchObservedRunningTime="2026-07-29 20:09:42.589946054 +0000 UTC m=+3148.085707396"
Jul 29 20:09:53.568183 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:53.568151    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:09:53.568675 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:09:53.568223    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:10:07.027257 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.027215    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"]
Jul 29 20:10:07.031863 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.031839    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.034351 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.034332    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-6488bc-openshift-default-dockercfg-vrhrd\""
Jul 29 20:10:07.044053 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.044025    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"]
Jul 29 20:10:07.083585 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.083555    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxrd4\" (UniqueName: \"kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-kube-api-access-fxrd4\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.083726 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.083592    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-envoy\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.083726 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.083611    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-certs\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.083726 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.083688    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istiod-ca-cert\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.083901 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.083761    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-socket\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.083901 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.083831    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-podinfo\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.083901 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.083863    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-data\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.084014 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.083933    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-token\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.084014 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.083957    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-credential-socket\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.184625 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.184591    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-token\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.184625 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.184630    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-credential-socket\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.184860 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.184657    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxrd4\" (UniqueName: \"kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-kube-api-access-fxrd4\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.184860 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.184689    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-envoy\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.184860 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.184706    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-certs\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.184860 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.184764    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istiod-ca-cert\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.184860 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.184815    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-socket\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.185171 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.184889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-podinfo\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.185171 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.184956    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-data\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.185171 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.185070    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-credential-socket\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.185308 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.185198    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-certs\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.185308 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.185263    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-socket\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.185575 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.185553    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-data\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.185734 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.185717    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istiod-ca-cert\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.186956 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.186941    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-envoy\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.187247 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.187229    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-podinfo\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.193366 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.193340    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-token\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.193615 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.193595    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxrd4\" (UniqueName: \"kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-kube-api-access-fxrd4\") pod \"e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") " pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.351055 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.350970    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:07.486725 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.486695    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"]
Jul 29 20:10:07.490082 ip-10-0-129-107 kubenswrapper[2566]: W0729 20:10:07.490054    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7dd319d_a266_4e3e_bebb_232bc8a9c303.slice/crio-91c90e4e050d8bcc0fe63cfebb2f16d85c902deaeb7da73826293f333fd923c6 WatchSource:0}: Error finding container 91c90e4e050d8bcc0fe63cfebb2f16d85c902deaeb7da73826293f333fd923c6: Status 404 returned error can't find the container with id 91c90e4e050d8bcc0fe63cfebb2f16d85c902deaeb7da73826293f333fd923c6
Jul 29 20:10:07.492177 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.492148    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 20:10:07.492251 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.492209    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 20:10:07.492251 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.492234    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 20:10:07.695233 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.695198    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" event={"ID":"b7dd319d-a266-4e3e-bebb-232bc8a9c303","Type":"ContainerStarted","Data":"c450427e1732603c188043f6315c6cb9bba6f2437142af3f6bbce88456b058ec"}
Jul 29 20:10:07.695399 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.695261    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" event={"ID":"b7dd319d-a266-4e3e-bebb-232bc8a9c303","Type":"ContainerStarted","Data":"91c90e4e050d8bcc0fe63cfebb2f16d85c902deaeb7da73826293f333fd923c6"}
Jul 29 20:10:07.717013 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:07.716958    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" podStartSLOduration=0.716940696 podStartE2EDuration="716.940696ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:10:07.71600289 +0000 UTC m=+3173.211764254" watchObservedRunningTime="2026-07-29 20:10:07.716940696 +0000 UTC m=+3173.212702041"
Jul 29 20:10:08.351247 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:08.351212    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:08.355747 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:08.355720    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:08.700380 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:08.700347    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:08.701361 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:08.701339    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:10.426264 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.426229    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"]
Jul 29 20:10:10.432196 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.432177    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:10:10.435040 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.435016    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-6488bc\""
Jul 29 20:10:10.435136 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.435059    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-6488bc-dockercfg-5grbh\""
Jul 29 20:10:10.441271 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.441243    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"]
Jul 29 20:10:10.520254 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.520218    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs8jv\" (UniqueName: \"kubernetes.io/projected/ad0207c7-82ae-47cd-ab01-a49f938a1db6-kube-api-access-qs8jv\") pod \"payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq\" (UID: \"ad0207c7-82ae-47cd-ab01-a49f938a1db6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:10:10.520437 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.520267    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad0207c7-82ae-47cd-ab01-a49f938a1db6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq\" (UID: \"ad0207c7-82ae-47cd-ab01-a49f938a1db6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:10:10.621161 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.621122    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs8jv\" (UniqueName: \"kubernetes.io/projected/ad0207c7-82ae-47cd-ab01-a49f938a1db6-kube-api-access-qs8jv\") pod \"payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq\" (UID: \"ad0207c7-82ae-47cd-ab01-a49f938a1db6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:10:10.621344 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.621169    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad0207c7-82ae-47cd-ab01-a49f938a1db6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq\" (UID: \"ad0207c7-82ae-47cd-ab01-a49f938a1db6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:10:10.621961 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.621943    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad0207c7-82ae-47cd-ab01-a49f938a1db6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq\" (UID: \"ad0207c7-82ae-47cd-ab01-a49f938a1db6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:10:10.629941 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.629918    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs8jv\" (UniqueName: \"kubernetes.io/projected/ad0207c7-82ae-47cd-ab01-a49f938a1db6-kube-api-access-qs8jv\") pod \"payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq\" (UID: \"ad0207c7-82ae-47cd-ab01-a49f938a1db6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:10:10.748915 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.748882    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:10:10.892171 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:10.892145    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"]
Jul 29 20:10:10.893591 ip-10-0-129-107 kubenswrapper[2566]: W0729 20:10:10.893559    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad0207c7_82ae_47cd_ab01_a49f938a1db6.slice/crio-df0870478a36d1d464a5209de604f653bef3988733d2e792f5b52b1b6547895c WatchSource:0}: Error finding container df0870478a36d1d464a5209de604f653bef3988733d2e792f5b52b1b6547895c: Status 404 returned error can't find the container with id df0870478a36d1d464a5209de604f653bef3988733d2e792f5b52b1b6547895c
Jul 29 20:10:11.719517 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:11.719473    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq" event={"ID":"ad0207c7-82ae-47cd-ab01-a49f938a1db6","Type":"ContainerStarted","Data":"6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751"}
Jul 29 20:10:11.719926 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:11.719545    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq" event={"ID":"ad0207c7-82ae-47cd-ab01-a49f938a1db6","Type":"ContainerStarted","Data":"df0870478a36d1d464a5209de604f653bef3988733d2e792f5b52b1b6547895c"}
Jul 29 20:10:11.719926 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:11.719662    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:10:11.736797 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:11.736748    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq" podStartSLOduration=1.736730951 podStartE2EDuration="1.736730951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:10:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:10:11.735072395 +0000 UTC m=+3177.230833752" watchObservedRunningTime="2026-07-29 20:10:11.736730951 +0000 UTC m=+3177.232492294"
Jul 29 20:10:22.727175 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:22.727141    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:10:27.811561 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:27.811528    2566 generic.go:356] "Generic (PLEG): container finished" podID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerID="c450427e1732603c188043f6315c6cb9bba6f2437142af3f6bbce88456b058ec" exitCode=0
Jul 29 20:10:27.812090 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:27.811603    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" event={"ID":"b7dd319d-a266-4e3e-bebb-232bc8a9c303","Type":"ContainerDied","Data":"c450427e1732603c188043f6315c6cb9bba6f2437142af3f6bbce88456b058ec"}
Jul 29 20:10:27.812164 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:27.812107    2566 scope.go:117] "RemoveContainer" containerID="c450427e1732603c188043f6315c6cb9bba6f2437142af3f6bbce88456b058ec"
Jul 29 20:10:27.813235 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:27.813209    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 20:10:27.813291 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:27.813274    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 20:10:27.813356 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:27.813343    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 20:10:28.818973 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:28.818935    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" event={"ID":"b7dd319d-a266-4e3e-bebb-232bc8a9c303","Type":"ContainerStarted","Data":"4a2a8833e83d543f1164019b765a05ac9a29da41b86967697003ad7806a8d2a9"}
Jul 29 20:10:29.351559 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:29.351514    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:29.351903 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:29.351857    2566 patch_prober.go:28] interesting pod/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 29 20:10:29.352013 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:29.351959    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 29 20:10:30.351747 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:30.351709    2566 patch_prober.go:28] interesting pod/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 29 20:10:30.352231 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:30.351780    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 29 20:10:31.352243 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:31.352203    2566 patch_prober.go:28] interesting pod/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 29 20:10:31.352853 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:31.352818    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 29 20:10:32.353291 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:32.352814    2566 patch_prober.go:28] interesting pod/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 29 20:10:32.353291 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:32.352890    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 29 20:10:33.355816 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:33.355785    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:33.356196 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:33.356061    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:33.356793 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:33.356772    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:10:54.874918 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.874883    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"]
Jul 29 20:10:54.879394 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.879374    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:54.883630 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.883603    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-fb0399\"/\"default-dockercfg-wfngm\""
Jul 29 20:10:54.883630 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.883618    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-fb0399\"/\"openshift-service-ca.crt\""
Jul 29 20:10:54.883817 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.883618    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-fb0399\"/\"kube-root-ca.crt\""
Jul 29 20:10:54.883817 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.883706    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-fb0399\"/\"ipp-route-fb0399-kserve-self-signed-certs\""
Jul 29 20:10:54.889342 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.889321    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"]
Jul 29 20:10:54.959300 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.959263    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef7edd40-3014-432b-98df-bda6d2f8b700-tls-certs\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:54.959489 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.959306    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb95t\" (UniqueName: \"kubernetes.io/projected/ef7edd40-3014-432b-98df-bda6d2f8b700-kube-api-access-nb95t\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:54.959489 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.959409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-model-cache\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:54.959489 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.959433    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-home\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:54.959623 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.959503    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-dshm\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:54.959623 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:54.959545    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-tmp-dir\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.061183 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.061145    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-model-cache\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.061390 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.061197    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-home\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.061390 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.061231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-dshm\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.061390 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.061268    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-tmp-dir\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.061390 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.061349    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef7edd40-3014-432b-98df-bda6d2f8b700-tls-certs\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.061390 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.061372    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb95t\" (UniqueName: \"kubernetes.io/projected/ef7edd40-3014-432b-98df-bda6d2f8b700-kube-api-access-nb95t\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.061738 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.061680    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-model-cache\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.061738 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.061716    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-home\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.061856 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.061780    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-tmp-dir\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.064261 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.064233    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-dshm\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.064763 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.064730    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef7edd40-3014-432b-98df-bda6d2f8b700-tls-certs\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.074378 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.074347    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb95t\" (UniqueName: \"kubernetes.io/projected/ef7edd40-3014-432b-98df-bda6d2f8b700-kube-api-access-nb95t\") pod \"ipp-route-fb0399-kserve-56664dff5f-ggpnr\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") " pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.190236 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.190214    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:10:55.327094 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.327057    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"]
Jul 29 20:10:55.328312 ip-10-0-129-107 kubenswrapper[2566]: W0729 20:10:55.328286    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef7edd40_3014_432b_98df_bda6d2f8b700.slice/crio-03b1fb91a3c0f89066f4781c77ed1e7d1acd4334fa7a1f6db8860f726c1784b7 WatchSource:0}: Error finding container 03b1fb91a3c0f89066f4781c77ed1e7d1acd4334fa7a1f6db8860f726c1784b7: Status 404 returned error can't find the container with id 03b1fb91a3c0f89066f4781c77ed1e7d1acd4334fa7a1f6db8860f726c1784b7
Jul 29 20:10:55.957907 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.957870    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr" event={"ID":"ef7edd40-3014-432b-98df-bda6d2f8b700","Type":"ContainerStarted","Data":"828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387"}
Jul 29 20:10:55.958370 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.957915    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr" event={"ID":"ef7edd40-3014-432b-98df-bda6d2f8b700","Type":"ContainerStarted","Data":"03b1fb91a3c0f89066f4781c77ed1e7d1acd4334fa7a1f6db8860f726c1784b7"}
Jul 29 20:10:55.984389 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:10:55.984339    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr" podStartSLOduration=1.9843230429999998 podStartE2EDuration="1.984323043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:10:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:10:55.982252334 +0000 UTC m=+3221.478013677" watchObservedRunningTime="2026-07-29 20:10:55.984323043 +0000 UTC m=+3221.480084384"
Jul 29 20:11:05.190887 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:05.190849    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:11:05.191383 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:05.190898    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:11:05.203559 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:05.203537    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:11:06.031175 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:06.031149    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:11:36.508849 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:36.508820    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924_833e778b-3ac4-4b31-939b-1694bfd028fe/payload-processing/0.log"
Jul 29 20:11:40.766975 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:40.766944    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"]
Jul 29 20:11:40.770777 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:40.770762    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:11:40.773235 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:40.773213    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-7720d1\""
Jul 29 20:11:40.773235 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:40.773228    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-7720d1-dockercfg-qd4qs\""
Jul 29 20:11:40.780425 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:40.780399    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"]
Jul 29 20:11:40.905445 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:40.905413    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kff7w\" (UniqueName: \"kubernetes.io/projected/939a1a9e-0093-4c24-9782-0afafa5a836a-kube-api-access-kff7w\") pod \"payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh\" (UID: \"939a1a9e-0093-4c24-9782-0afafa5a836a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:11:40.905647 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:40.905468    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/939a1a9e-0093-4c24-9782-0afafa5a836a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh\" (UID: \"939a1a9e-0093-4c24-9782-0afafa5a836a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:11:41.006182 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:41.006145    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kff7w\" (UniqueName: \"kubernetes.io/projected/939a1a9e-0093-4c24-9782-0afafa5a836a-kube-api-access-kff7w\") pod \"payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh\" (UID: \"939a1a9e-0093-4c24-9782-0afafa5a836a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:11:41.006356 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:41.006189    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/939a1a9e-0093-4c24-9782-0afafa5a836a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh\" (UID: \"939a1a9e-0093-4c24-9782-0afafa5a836a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:11:41.006922 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:41.006903    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/939a1a9e-0093-4c24-9782-0afafa5a836a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh\" (UID: \"939a1a9e-0093-4c24-9782-0afafa5a836a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:11:41.021063 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:41.021003    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kff7w\" (UniqueName: \"kubernetes.io/projected/939a1a9e-0093-4c24-9782-0afafa5a836a-kube-api-access-kff7w\") pod \"payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh\" (UID: \"939a1a9e-0093-4c24-9782-0afafa5a836a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:11:41.082308 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:41.082279    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:11:41.232092 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:41.232057    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"]
Jul 29 20:11:41.234513 ip-10-0-129-107 kubenswrapper[2566]: W0729 20:11:41.233920    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod939a1a9e_0093_4c24_9782_0afafa5a836a.slice/crio-5e5e67fab584eb7b74edde12627d04755b57ff7f8be6858050a0b63e368fe420 WatchSource:0}: Error finding container 5e5e67fab584eb7b74edde12627d04755b57ff7f8be6858050a0b63e368fe420: Status 404 returned error can't find the container with id 5e5e67fab584eb7b74edde12627d04755b57ff7f8be6858050a0b63e368fe420
Jul 29 20:11:42.209375 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:42.209341    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh" event={"ID":"939a1a9e-0093-4c24-9782-0afafa5a836a","Type":"ContainerStarted","Data":"38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e"}
Jul 29 20:11:42.209853 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:42.209394    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh" event={"ID":"939a1a9e-0093-4c24-9782-0afafa5a836a","Type":"ContainerStarted","Data":"5e5e67fab584eb7b74edde12627d04755b57ff7f8be6858050a0b63e368fe420"}
Jul 29 20:11:42.209853 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:42.209486    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:11:42.232582 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:42.232535    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh" podStartSLOduration=2.23252039 podStartE2EDuration="2.23252039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:11:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:11:42.230701181 +0000 UTC m=+3267.726462520" watchObservedRunningTime="2026-07-29 20:11:42.23252039 +0000 UTC m=+3267.728281733"
Jul 29 20:11:53.217634 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:11:53.217601    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:12:09.533898 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:09.533862    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"]
Jul 29 20:12:09.534540 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:09.534081    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh" podUID="939a1a9e-0093-4c24-9782-0afafa5a836a" containerName="payload-pre-processing" containerID="cri-o://38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e" gracePeriod=30
Jul 29 20:12:09.790749 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:09.790684    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:12:09.884399 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:09.884357    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kff7w\" (UniqueName: \"kubernetes.io/projected/939a1a9e-0093-4c24-9782-0afafa5a836a-kube-api-access-kff7w\") pod \"939a1a9e-0093-4c24-9782-0afafa5a836a\" (UID: \"939a1a9e-0093-4c24-9782-0afafa5a836a\") "
Jul 29 20:12:09.884613 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:09.884413    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/939a1a9e-0093-4c24-9782-0afafa5a836a-plugins-config-volume\") pod \"939a1a9e-0093-4c24-9782-0afafa5a836a\" (UID: \"939a1a9e-0093-4c24-9782-0afafa5a836a\") "
Jul 29 20:12:09.884863 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:09.884837    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/939a1a9e-0093-4c24-9782-0afafa5a836a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "939a1a9e-0093-4c24-9782-0afafa5a836a" (UID: "939a1a9e-0093-4c24-9782-0afafa5a836a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:12:09.886529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:09.886504    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/939a1a9e-0093-4c24-9782-0afafa5a836a-kube-api-access-kff7w" (OuterVolumeSpecName: "kube-api-access-kff7w") pod "939a1a9e-0093-4c24-9782-0afafa5a836a" (UID: "939a1a9e-0093-4c24-9782-0afafa5a836a"). InnerVolumeSpecName "kube-api-access-kff7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:12:09.985701 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:09.985667    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kff7w\" (UniqueName: \"kubernetes.io/projected/939a1a9e-0093-4c24-9782-0afafa5a836a-kube-api-access-kff7w\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:09.985701 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:09.985697    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/939a1a9e-0093-4c24-9782-0afafa5a836a-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:10.357558 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:10.357523    2566 generic.go:356] "Generic (PLEG): container finished" podID="939a1a9e-0093-4c24-9782-0afafa5a836a" containerID="38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e" exitCode=0
Jul 29 20:12:10.357744 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:10.357572    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"
Jul 29 20:12:10.357744 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:10.357608    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh" event={"ID":"939a1a9e-0093-4c24-9782-0afafa5a836a","Type":"ContainerDied","Data":"38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e"}
Jul 29 20:12:10.357744 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:10.357660    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh" event={"ID":"939a1a9e-0093-4c24-9782-0afafa5a836a","Type":"ContainerDied","Data":"5e5e67fab584eb7b74edde12627d04755b57ff7f8be6858050a0b63e368fe420"}
Jul 29 20:12:10.357744 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:10.357682    2566 scope.go:117] "RemoveContainer" containerID="38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e"
Jul 29 20:12:10.367131 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:10.367113    2566 scope.go:117] "RemoveContainer" containerID="38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e"
Jul 29 20:12:10.367384 ip-10-0-129-107 kubenswrapper[2566]: E0729 20:12:10.367364    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e\": container with ID starting with 38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e not found: ID does not exist" containerID="38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e"
Jul 29 20:12:10.367463 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:10.367394    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e"} err="failed to get container status \"38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e\": rpc error: code = NotFound desc = could not find container \"38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e\": container with ID starting with 38f5d6ca7718e0f9ecbb4f81ca00bc1e57d51e97bafa412559554526c1d2493e not found: ID does not exist"
Jul 29 20:12:10.382654 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:10.382628    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"]
Jul 29 20:12:10.388314 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:10.388293    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7720d1-6f88f6c764-lc6jh"]
Jul 29 20:12:11.101950 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:11.101914    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="939a1a9e-0093-4c24-9782-0afafa5a836a" path="/var/lib/kubelet/pods/939a1a9e-0093-4c24-9782-0afafa5a836a/volumes"
Jul 29 20:12:35.378987 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.378953    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"]
Jul 29 20:12:35.379484 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.379206    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924" podUID="833e778b-3ac4-4b31-939b-1694bfd028fe" containerName="payload-processing" containerID="cri-o://bf5c1a1e8070fd07d04564bd24391de572c8eeb40e353daddbd1d07b19b73416" gracePeriod=30
Jul 29 20:12:35.391261 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.391236    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"]
Jul 29 20:12:35.391416 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.391396    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf" podUID="51a53f77-9600-44b1-be67-1c556b362c01" containerName="payload-pre-processing" containerID="cri-o://6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877" gracePeriod=30
Jul 29 20:12:35.501098 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.501069    2566 generic.go:356] "Generic (PLEG): container finished" podID="833e778b-3ac4-4b31-939b-1694bfd028fe" containerID="bf5c1a1e8070fd07d04564bd24391de572c8eeb40e353daddbd1d07b19b73416" exitCode=0
Jul 29 20:12:35.501249 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.501162    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924" event={"ID":"833e778b-3ac4-4b31-939b-1694bfd028fe","Type":"ContainerDied","Data":"bf5c1a1e8070fd07d04564bd24391de572c8eeb40e353daddbd1d07b19b73416"}
Jul 29 20:12:35.648435 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.648410    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:12:35.685998 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.685975    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:12:35.752642 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.752608    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a53f77-9600-44b1-be67-1c556b362c01-plugins-config-volume\") pod \"51a53f77-9600-44b1-be67-1c556b362c01\" (UID: \"51a53f77-9600-44b1-be67-1c556b362c01\") "
Jul 29 20:12:35.752917 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.752658    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8f2m\" (UniqueName: \"kubernetes.io/projected/51a53f77-9600-44b1-be67-1c556b362c01-kube-api-access-l8f2m\") pod \"51a53f77-9600-44b1-be67-1c556b362c01\" (UID: \"51a53f77-9600-44b1-be67-1c556b362c01\") "
Jul 29 20:12:35.752917 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.752707    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/833e778b-3ac4-4b31-939b-1694bfd028fe-plugins-config-volume\") pod \"833e778b-3ac4-4b31-939b-1694bfd028fe\" (UID: \"833e778b-3ac4-4b31-939b-1694bfd028fe\") "
Jul 29 20:12:35.752917 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.752769    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nvzz\" (UniqueName: \"kubernetes.io/projected/833e778b-3ac4-4b31-939b-1694bfd028fe-kube-api-access-6nvzz\") pod \"833e778b-3ac4-4b31-939b-1694bfd028fe\" (UID: \"833e778b-3ac4-4b31-939b-1694bfd028fe\") "
Jul 29 20:12:35.753100 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.753051    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51a53f77-9600-44b1-be67-1c556b362c01-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "51a53f77-9600-44b1-be67-1c556b362c01" (UID: "51a53f77-9600-44b1-be67-1c556b362c01"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:12:35.753100 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.753053    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/833e778b-3ac4-4b31-939b-1694bfd028fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "833e778b-3ac4-4b31-939b-1694bfd028fe" (UID: "833e778b-3ac4-4b31-939b-1694bfd028fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:12:35.753193 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.753159    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a53f77-9600-44b1-be67-1c556b362c01-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:35.753193 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.753173    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/833e778b-3ac4-4b31-939b-1694bfd028fe-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:35.754780 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.754757    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/833e778b-3ac4-4b31-939b-1694bfd028fe-kube-api-access-6nvzz" (OuterVolumeSpecName: "kube-api-access-6nvzz") pod "833e778b-3ac4-4b31-939b-1694bfd028fe" (UID: "833e778b-3ac4-4b31-939b-1694bfd028fe"). InnerVolumeSpecName "kube-api-access-6nvzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:12:35.754780 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.754768    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51a53f77-9600-44b1-be67-1c556b362c01-kube-api-access-l8f2m" (OuterVolumeSpecName: "kube-api-access-l8f2m") pod "51a53f77-9600-44b1-be67-1c556b362c01" (UID: "51a53f77-9600-44b1-be67-1c556b362c01"). InnerVolumeSpecName "kube-api-access-l8f2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:12:35.854265 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.854236    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nvzz\" (UniqueName: \"kubernetes.io/projected/833e778b-3ac4-4b31-939b-1694bfd028fe-kube-api-access-6nvzz\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:35.854265 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:35.854265    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8f2m\" (UniqueName: \"kubernetes.io/projected/51a53f77-9600-44b1-be67-1c556b362c01-kube-api-access-l8f2m\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:36.507362 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.507326    2566 generic.go:356] "Generic (PLEG): container finished" podID="51a53f77-9600-44b1-be67-1c556b362c01" containerID="6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877" exitCode=0
Jul 29 20:12:36.507817 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.507376    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"
Jul 29 20:12:36.507817 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.507407    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf" event={"ID":"51a53f77-9600-44b1-be67-1c556b362c01","Type":"ContainerDied","Data":"6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877"}
Jul 29 20:12:36.507817 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.507478    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf" event={"ID":"51a53f77-9600-44b1-be67-1c556b362c01","Type":"ContainerDied","Data":"835060751fab081403ed1e1ace63e67d80f26d8535f48cc3fb5be99672239406"}
Jul 29 20:12:36.507817 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.507513    2566 scope.go:117] "RemoveContainer" containerID="6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877"
Jul 29 20:12:36.508832 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.508815    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"
Jul 29 20:12:36.508919 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.508844    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924" event={"ID":"833e778b-3ac4-4b31-939b-1694bfd028fe","Type":"ContainerDied","Data":"6cd0cdd80e6363c86246ff7594c26ccccd71ad1dddec2ee1561b284096333299"}
Jul 29 20:12:36.516491 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.516476    2566 scope.go:117] "RemoveContainer" containerID="6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877"
Jul 29 20:12:36.516865 ip-10-0-129-107 kubenswrapper[2566]: E0729 20:12:36.516747    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877\": container with ID starting with 6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877 not found: ID does not exist" containerID="6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877"
Jul 29 20:12:36.516865 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.516773    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877"} err="failed to get container status \"6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877\": rpc error: code = NotFound desc = could not find container \"6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877\": container with ID starting with 6432227b288df8d4a303bff039c006bf71f4bacf85dc7cb06e2add7a0497e877 not found: ID does not exist"
Jul 29 20:12:36.516865 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.516800    2566 scope.go:117] "RemoveContainer" containerID="bf5c1a1e8070fd07d04564bd24391de572c8eeb40e353daddbd1d07b19b73416"
Jul 29 20:12:36.538604 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.538575    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"]
Jul 29 20:12:36.545877 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.545846    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb0399-785f5849f6-z6vtf"]
Jul 29 20:12:36.556381 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.556361    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"]
Jul 29 20:12:36.561369 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:36.561351    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb0399-68bbd99f48-th924"]
Jul 29 20:12:37.102410 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:37.102378    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51a53f77-9600-44b1-be67-1c556b362c01" path="/var/lib/kubelet/pods/51a53f77-9600-44b1-be67-1c556b362c01/volumes"
Jul 29 20:12:37.102794 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:37.102779    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="833e778b-3ac4-4b31-939b-1694bfd028fe" path="/var/lib/kubelet/pods/833e778b-3ac4-4b31-939b-1694bfd028fe/volumes"
Jul 29 20:12:40.944564 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:40.944524    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"]
Jul 29 20:12:55.964432 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:55.964359    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr" podUID="ef7edd40-3014-432b-98df-bda6d2f8b700" containerName="main" containerID="cri-o://828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387" gracePeriod=45
Jul 29 20:12:56.020985 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.020943    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr" podUID="ef7edd40-3014-432b-98df-bda6d2f8b700" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": dial tcp 10.134.0.70:8000: connect: connection refused"
Jul 29 20:12:56.224522 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.224440    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:12:56.359840 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.359808    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-tmp-dir\") pod \"ef7edd40-3014-432b-98df-bda6d2f8b700\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") "
Jul 29 20:12:56.359840 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.359843    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-dshm\") pod \"ef7edd40-3014-432b-98df-bda6d2f8b700\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") "
Jul 29 20:12:56.360074 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.359951    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-model-cache\") pod \"ef7edd40-3014-432b-98df-bda6d2f8b700\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") "
Jul 29 20:12:56.360074 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.360017    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nb95t\" (UniqueName: \"kubernetes.io/projected/ef7edd40-3014-432b-98df-bda6d2f8b700-kube-api-access-nb95t\") pod \"ef7edd40-3014-432b-98df-bda6d2f8b700\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") "
Jul 29 20:12:56.360074 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.360039    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-home\") pod \"ef7edd40-3014-432b-98df-bda6d2f8b700\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") "
Jul 29 20:12:56.360074 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.360059    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef7edd40-3014-432b-98df-bda6d2f8b700-tls-certs\") pod \"ef7edd40-3014-432b-98df-bda6d2f8b700\" (UID: \"ef7edd40-3014-432b-98df-bda6d2f8b700\") "
Jul 29 20:12:56.360295 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.360145    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ef7edd40-3014-432b-98df-bda6d2f8b700" (UID: "ef7edd40-3014-432b-98df-bda6d2f8b700"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:12:56.360295 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.360232    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-model-cache" (OuterVolumeSpecName: "model-cache") pod "ef7edd40-3014-432b-98df-bda6d2f8b700" (UID: "ef7edd40-3014-432b-98df-bda6d2f8b700"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:12:56.360406 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.360310    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-home" (OuterVolumeSpecName: "home") pod "ef7edd40-3014-432b-98df-bda6d2f8b700" (UID: "ef7edd40-3014-432b-98df-bda6d2f8b700"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:12:56.360492 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.360412    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-model-cache\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:56.360492 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.360424    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-home\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:56.360492 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.360434    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-tmp-dir\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:56.362288 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.362258    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-dshm" (OuterVolumeSpecName: "dshm") pod "ef7edd40-3014-432b-98df-bda6d2f8b700" (UID: "ef7edd40-3014-432b-98df-bda6d2f8b700"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:12:56.362288 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.362276    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef7edd40-3014-432b-98df-bda6d2f8b700-kube-api-access-nb95t" (OuterVolumeSpecName: "kube-api-access-nb95t") pod "ef7edd40-3014-432b-98df-bda6d2f8b700" (UID: "ef7edd40-3014-432b-98df-bda6d2f8b700"). InnerVolumeSpecName "kube-api-access-nb95t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:12:56.362288 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.362262    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef7edd40-3014-432b-98df-bda6d2f8b700-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ef7edd40-3014-432b-98df-bda6d2f8b700" (UID: "ef7edd40-3014-432b-98df-bda6d2f8b700"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:12:56.461074 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.461029    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef7edd40-3014-432b-98df-bda6d2f8b700-dshm\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:56.461074 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.461067    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nb95t\" (UniqueName: \"kubernetes.io/projected/ef7edd40-3014-432b-98df-bda6d2f8b700-kube-api-access-nb95t\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:56.461074 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.461079    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef7edd40-3014-432b-98df-bda6d2f8b700-tls-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:12:56.615805 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.615708    2566 generic.go:356] "Generic (PLEG): container finished" podID="ef7edd40-3014-432b-98df-bda6d2f8b700" containerID="828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387" exitCode=0
Jul 29 20:12:56.615805 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.615761    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"
Jul 29 20:12:56.615995 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.615801    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr" event={"ID":"ef7edd40-3014-432b-98df-bda6d2f8b700","Type":"ContainerDied","Data":"828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387"}
Jul 29 20:12:56.615995 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.615845    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr" event={"ID":"ef7edd40-3014-432b-98df-bda6d2f8b700","Type":"ContainerDied","Data":"03b1fb91a3c0f89066f4781c77ed1e7d1acd4334fa7a1f6db8860f726c1784b7"}
Jul 29 20:12:56.615995 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.615864    2566 scope.go:117] "RemoveContainer" containerID="828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387"
Jul 29 20:12:56.625373 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.625356    2566 scope.go:117] "RemoveContainer" containerID="828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387"
Jul 29 20:12:56.625649 ip-10-0-129-107 kubenswrapper[2566]: E0729 20:12:56.625630    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387\": container with ID starting with 828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387 not found: ID does not exist" containerID="828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387"
Jul 29 20:12:56.625721 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.625656    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387"} err="failed to get container status \"828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387\": rpc error: code = NotFound desc = could not find container \"828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387\": container with ID starting with 828b15d5f551b7fa36ad0f2701a50f4617f274f80a17ec0124a13641deb48387 not found: ID does not exist"
Jul 29 20:12:56.639118 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.639095    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"]
Jul 29 20:12:56.645090 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:56.645058    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-fb0399/ipp-route-fb0399-kserve-56664dff5f-ggpnr"]
Jul 29 20:12:57.103029 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:12:57.102987    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef7edd40-3014-432b-98df-bda6d2f8b700" path="/var/lib/kubelet/pods/ef7edd40-3014-432b-98df-bda6d2f8b700/volumes"
Jul 29 20:13:38.536413 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.536375    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"]
Jul 29 20:13:38.536938 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.536628    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq" podUID="ad0207c7-82ae-47cd-ab01-a49f938a1db6" containerName="payload-pre-processing" containerID="cri-o://6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751" gracePeriod=30
Jul 29 20:13:38.806797 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.806770    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:13:38.840845 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.840806    2566 generic.go:356] "Generic (PLEG): container finished" podID="ad0207c7-82ae-47cd-ab01-a49f938a1db6" containerID="6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751" exitCode=0
Jul 29 20:13:38.840845 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.840845    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"
Jul 29 20:13:38.841087 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.840887    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq" event={"ID":"ad0207c7-82ae-47cd-ab01-a49f938a1db6","Type":"ContainerDied","Data":"6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751"}
Jul 29 20:13:38.841087 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.840952    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq" event={"ID":"ad0207c7-82ae-47cd-ab01-a49f938a1db6","Type":"ContainerDied","Data":"df0870478a36d1d464a5209de604f653bef3988733d2e792f5b52b1b6547895c"}
Jul 29 20:13:38.841087 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.840979    2566 scope.go:117] "RemoveContainer" containerID="6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751"
Jul 29 20:13:38.849983 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.849961    2566 scope.go:117] "RemoveContainer" containerID="6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751"
Jul 29 20:13:38.850256 ip-10-0-129-107 kubenswrapper[2566]: E0729 20:13:38.850237    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751\": container with ID starting with 6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751 not found: ID does not exist" containerID="6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751"
Jul 29 20:13:38.850321 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.850266    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751"} err="failed to get container status \"6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751\": rpc error: code = NotFound desc = could not find container \"6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751\": container with ID starting with 6591e222734fb4c7a25f5b9e6178507faafcaf3497032b30fbbcf2227e0bb751 not found: ID does not exist"
Jul 29 20:13:38.893937 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.893905    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad0207c7-82ae-47cd-ab01-a49f938a1db6-plugins-config-volume\") pod \"ad0207c7-82ae-47cd-ab01-a49f938a1db6\" (UID: \"ad0207c7-82ae-47cd-ab01-a49f938a1db6\") "
Jul 29 20:13:38.894092 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.893964    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs8jv\" (UniqueName: \"kubernetes.io/projected/ad0207c7-82ae-47cd-ab01-a49f938a1db6-kube-api-access-qs8jv\") pod \"ad0207c7-82ae-47cd-ab01-a49f938a1db6\" (UID: \"ad0207c7-82ae-47cd-ab01-a49f938a1db6\") "
Jul 29 20:13:38.894279 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.894254    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad0207c7-82ae-47cd-ab01-a49f938a1db6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ad0207c7-82ae-47cd-ab01-a49f938a1db6" (UID: "ad0207c7-82ae-47cd-ab01-a49f938a1db6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:13:38.896107 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.896086    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad0207c7-82ae-47cd-ab01-a49f938a1db6-kube-api-access-qs8jv" (OuterVolumeSpecName: "kube-api-access-qs8jv") pod "ad0207c7-82ae-47cd-ab01-a49f938a1db6" (UID: "ad0207c7-82ae-47cd-ab01-a49f938a1db6"). InnerVolumeSpecName "kube-api-access-qs8jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:13:38.995155 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.995127    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad0207c7-82ae-47cd-ab01-a49f938a1db6-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:13:38.995155 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:38.995151    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs8jv\" (UniqueName: \"kubernetes.io/projected/ad0207c7-82ae-47cd-ab01-a49f938a1db6-kube-api-access-qs8jv\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:13:39.164294 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:39.164255    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"]
Jul 29 20:13:39.168647 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:39.168621    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-6488bc-55d7ddc948-8t4jq"]
Jul 29 20:13:41.102445 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:41.102411    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad0207c7-82ae-47cd-ab01-a49f938a1db6" path="/var/lib/kubelet/pods/ad0207c7-82ae-47cd-ab01-a49f938a1db6/volumes"
Jul 29 20:13:45.728517 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:45.728479    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"]
Jul 29 20:13:45.729014 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:45.728747    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy" containerID="cri-o://4a2a8833e83d543f1164019b765a05ac9a29da41b86967697003ad7806a8d2a9" gracePeriod=30
Jul 29 20:13:48.359216 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:48.359176    2566 patch_prober.go:28] interesting pod/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 20:13:48.359704 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:48.359257    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 20:13:50.908776 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:50.908734    2566 generic.go:356] "Generic (PLEG): container finished" podID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerID="4a2a8833e83d543f1164019b765a05ac9a29da41b86967697003ad7806a8d2a9" exitCode=0
Jul 29 20:13:50.909195 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:50.908804    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" event={"ID":"b7dd319d-a266-4e3e-bebb-232bc8a9c303","Type":"ContainerDied","Data":"4a2a8833e83d543f1164019b765a05ac9a29da41b86967697003ad7806a8d2a9"}
Jul 29 20:13:50.909195 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:50.908871    2566 scope.go:117] "RemoveContainer" containerID="c450427e1732603c188043f6315c6cb9bba6f2437142af3f6bbce88456b058ec"
Jul 29 20:13:50.989432 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:50.989408    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:13:51.116529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.114317    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-token\") pod \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") "
Jul 29 20:13:51.116529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.114385    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-credential-socket\") pod \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") "
Jul 29 20:13:51.116529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.114420    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-data\") pod \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") "
Jul 29 20:13:51.116529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.114445    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istiod-ca-cert\") pod \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") "
Jul 29 20:13:51.116529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.114504    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-certs\") pod \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") "
Jul 29 20:13:51.116529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.114564    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-podinfo\") pod \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") "
Jul 29 20:13:51.116529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.114595    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-envoy\") pod \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") "
Jul 29 20:13:51.116529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.114695    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-socket\") pod \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") "
Jul 29 20:13:51.116529 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.114782    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxrd4\" (UniqueName: \"kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-kube-api-access-fxrd4\") pod \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\" (UID: \"b7dd319d-a266-4e3e-bebb-232bc8a9c303\") "
Jul 29 20:13:51.121366 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.121289    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7dd319d-a266-4e3e-bebb-232bc8a9c303" (UID: "b7dd319d-a266-4e3e-bebb-232bc8a9c303"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:13:51.121705 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.121593    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7dd319d-a266-4e3e-bebb-232bc8a9c303" (UID: "b7dd319d-a266-4e3e-bebb-232bc8a9c303"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:13:51.121705 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.121603    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7dd319d-a266-4e3e-bebb-232bc8a9c303" (UID: "b7dd319d-a266-4e3e-bebb-232bc8a9c303"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:13:51.122016 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.121970    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-kube-api-access-fxrd4" (OuterVolumeSpecName: "kube-api-access-fxrd4") pod "b7dd319d-a266-4e3e-bebb-232bc8a9c303" (UID: "b7dd319d-a266-4e3e-bebb-232bc8a9c303"). InnerVolumeSpecName "kube-api-access-fxrd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:13:51.122126 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.121988    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7dd319d-a266-4e3e-bebb-232bc8a9c303" (UID: "b7dd319d-a266-4e3e-bebb-232bc8a9c303"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:13:51.122229 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.122205    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7dd319d-a266-4e3e-bebb-232bc8a9c303" (UID: "b7dd319d-a266-4e3e-bebb-232bc8a9c303"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:13:51.123590 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.123567    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7dd319d-a266-4e3e-bebb-232bc8a9c303" (UID: "b7dd319d-a266-4e3e-bebb-232bc8a9c303"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:13:51.123680 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.123596    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7dd319d-a266-4e3e-bebb-232bc8a9c303" (UID: "b7dd319d-a266-4e3e-bebb-232bc8a9c303"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:13:51.123814 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.123799    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7dd319d-a266-4e3e-bebb-232bc8a9c303" (UID: "b7dd319d-a266-4e3e-bebb-232bc8a9c303"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 20:13:51.216749 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.216709    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxrd4\" (UniqueName: \"kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-kube-api-access-fxrd4\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:13:51.216749 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.216744    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-token\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:13:51.216749 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.216754    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-credential-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:13:51.216986 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.216763    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-data\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:13:51.216986 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.216771    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istiod-ca-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:13:51.216986 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.216794    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:13:51.216986 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.216802    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-podinfo\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:13:51.216986 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.216809    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-istio-envoy\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:13:51.216986 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.216817    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dd319d-a266-4e3e-bebb-232bc8a9c303-workload-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:13:51.915674 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.915643    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"
Jul 29 20:13:51.918518 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.918476    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts" event={"ID":"b7dd319d-a266-4e3e-bebb-232bc8a9c303","Type":"ContainerDied","Data":"91c90e4e050d8bcc0fe63cfebb2f16d85c902deaeb7da73826293f333fd923c6"}
Jul 29 20:13:51.918659 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.918585    2566 scope.go:117] "RemoveContainer" containerID="4a2a8833e83d543f1164019b765a05ac9a29da41b86967697003ad7806a8d2a9"
Jul 29 20:13:51.945075 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.945046    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"]
Jul 29 20:13:51.946822 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:51.946800    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-6488bc-openshift-default-dd7c889d-ptsts"]
Jul 29 20:13:53.101805 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:13:53.101769    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" path="/var/lib/kubelet/pods/b7dd319d-a266-4e3e-bebb-232bc8a9c303/volumes"
Jul 29 20:14:03.928135 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:03.928096    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"]
Jul 29 20:14:03.929118 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:03.929058    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerName="istio-proxy" containerID="cri-o://dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30" gracePeriod=30
Jul 29 20:14:09.189711 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.189686    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 20:14:09.294904 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.294822    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istiod-ca-cert\") pod \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") "
Jul 29 20:14:09.294904 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.294857    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-token\") pod \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") "
Jul 29 20:14:09.295092 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.294911    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-credential-socket\") pod \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") "
Jul 29 20:14:09.295092 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.294937    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-data\") pod \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") "
Jul 29 20:14:09.295092 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.294977    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-envoy\") pod \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") "
Jul 29 20:14:09.295092 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295067    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-podinfo\") pod \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") "
Jul 29 20:14:09.295282 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295109    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-certs\") pod \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") "
Jul 29 20:14:09.295282 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295165    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98l46\" (UniqueName: \"kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-kube-api-access-98l46\") pod \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") "
Jul 29 20:14:09.295282 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295193    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-socket\") pod \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\" (UID: \"91b331f4-313d-4c17-b3cc-d4b66ab01bc6\") "
Jul 29 20:14:09.295282 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295269    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "91b331f4-313d-4c17-b3cc-d4b66ab01bc6" (UID: "91b331f4-313d-4c17-b3cc-d4b66ab01bc6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:14:09.295549 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295282    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "91b331f4-313d-4c17-b3cc-d4b66ab01bc6" (UID: "91b331f4-313d-4c17-b3cc-d4b66ab01bc6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:14:09.295549 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295500    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "91b331f4-313d-4c17-b3cc-d4b66ab01bc6" (UID: "91b331f4-313d-4c17-b3cc-d4b66ab01bc6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:14:09.295665 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295590    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "91b331f4-313d-4c17-b3cc-d4b66ab01bc6" (UID: "91b331f4-313d-4c17-b3cc-d4b66ab01bc6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:14:09.295724 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295681    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-certs\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:14:09.295724 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295700    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-workload-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:14:09.295724 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295714    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istiod-ca-cert\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:14:09.295881 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295727    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-credential-socket\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:14:09.295881 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.295868    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-data" (OuterVolumeSpecName: "istio-data") pod "91b331f4-313d-4c17-b3cc-d4b66ab01bc6" (UID: "91b331f4-313d-4c17-b3cc-d4b66ab01bc6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:14:09.297181 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.297153    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-token" (OuterVolumeSpecName: "istio-token") pod "91b331f4-313d-4c17-b3cc-d4b66ab01bc6" (UID: "91b331f4-313d-4c17-b3cc-d4b66ab01bc6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:14:09.297303 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.297219    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-kube-api-access-98l46" (OuterVolumeSpecName: "kube-api-access-98l46") pod "91b331f4-313d-4c17-b3cc-d4b66ab01bc6" (UID: "91b331f4-313d-4c17-b3cc-d4b66ab01bc6"). InnerVolumeSpecName "kube-api-access-98l46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:14:09.297375 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.297360    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "91b331f4-313d-4c17-b3cc-d4b66ab01bc6" (UID: "91b331f4-313d-4c17-b3cc-d4b66ab01bc6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:14:09.297414 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.297382    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "91b331f4-313d-4c17-b3cc-d4b66ab01bc6" (UID: "91b331f4-313d-4c17-b3cc-d4b66ab01bc6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 20:14:09.396418 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.396381    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98l46\" (UniqueName: \"kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-kube-api-access-98l46\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:14:09.396418 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.396417    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-token\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:14:09.396418 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.396427    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-data\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:14:09.396692 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.396435    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-envoy\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:14:09.396692 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:09.396443    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91b331f4-313d-4c17-b3cc-d4b66ab01bc6-istio-podinfo\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:14:10.007195 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.007156    2566 generic.go:356] "Generic (PLEG): container finished" podID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerID="dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30" exitCode=0
Jul 29 20:14:10.007378 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.007230    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"
Jul 29 20:14:10.007378 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.007239    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" event={"ID":"91b331f4-313d-4c17-b3cc-d4b66ab01bc6","Type":"ContainerDied","Data":"dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30"}
Jul 29 20:14:10.007378 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.007320    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7" event={"ID":"91b331f4-313d-4c17-b3cc-d4b66ab01bc6","Type":"ContainerDied","Data":"0b5bc69440d9c7fb9b9018f4c59be555560e7674bc483aa5dd31c6715bad76a8"}
Jul 29 20:14:10.007378 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.007343    2566 scope.go:117] "RemoveContainer" containerID="dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30"
Jul 29 20:14:10.017262 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.017242    2566 scope.go:117] "RemoveContainer" containerID="118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027"
Jul 29 20:14:10.027538 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.027518    2566 scope.go:117] "RemoveContainer" containerID="dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30"
Jul 29 20:14:10.027789 ip-10-0-129-107 kubenswrapper[2566]: E0729 20:14:10.027767    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30\": container with ID starting with dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30 not found: ID does not exist" containerID="dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30"
Jul 29 20:14:10.027839 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.027799    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30"} err="failed to get container status \"dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30\": rpc error: code = NotFound desc = could not find container \"dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30\": container with ID starting with dc834178bcce27f68c9933dd6ed21151f3d5148bb85bdf461ab41d6a51536b30 not found: ID does not exist"
Jul 29 20:14:10.027839 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.027818    2566 scope.go:117] "RemoveContainer" containerID="118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027"
Jul 29 20:14:10.028074 ip-10-0-129-107 kubenswrapper[2566]: E0729 20:14:10.028058    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027\": container with ID starting with 118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027 not found: ID does not exist" containerID="118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027"
Jul 29 20:14:10.028145 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.028078    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027"} err="failed to get container status \"118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027\": rpc error: code = NotFound desc = could not find container \"118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027\": container with ID starting with 118217cb9501be14f0ebf1a6782609ee28acd8296c456811c23000b7ad526027 not found: ID does not exist"
Jul 29 20:14:10.031248 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.031225    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"]
Jul 29 20:14:10.037614 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:10.037593    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a60880-openshift-default-6bf448584-khph7"]
Jul 29 20:14:11.101871 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:11.101839    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" path="/var/lib/kubelet/pods/91b331f4-313d-4c17-b3cc-d4b66ab01bc6/volumes"
Jul 29 20:14:14.956046 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:14.956013    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"]
Jul 29 20:14:14.956525 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:14.956323    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb" podUID="1a8c94cf-e6c7-4366-b180-c3db7b37b16e" containerName="payload-pre-processing" containerID="cri-o://bb8ee236a76fe6671e5ee30b44754b7ede3da08b3e21e5e2fedcbfe0d8305ad0" gracePeriod=30
Jul 29 20:14:15.206533 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:15.206445    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 20:14:15.252806 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:15.252770    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-plugins-config-volume\") pod \"1a8c94cf-e6c7-4366-b180-c3db7b37b16e\" (UID: \"1a8c94cf-e6c7-4366-b180-c3db7b37b16e\") "
Jul 29 20:14:15.252989 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:15.252813    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5w82l\" (UniqueName: \"kubernetes.io/projected/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-kube-api-access-5w82l\") pod \"1a8c94cf-e6c7-4366-b180-c3db7b37b16e\" (UID: \"1a8c94cf-e6c7-4366-b180-c3db7b37b16e\") "
Jul 29 20:14:15.253104 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:15.253081    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a8c94cf-e6c7-4366-b180-c3db7b37b16e" (UID: "1a8c94cf-e6c7-4366-b180-c3db7b37b16e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:14:15.254880 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:15.254858    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-kube-api-access-5w82l" (OuterVolumeSpecName: "kube-api-access-5w82l") pod "1a8c94cf-e6c7-4366-b180-c3db7b37b16e" (UID: "1a8c94cf-e6c7-4366-b180-c3db7b37b16e"). InnerVolumeSpecName "kube-api-access-5w82l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:14:15.357574 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:15.357517    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-plugins-config-volume\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:14:15.357574 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:15.357553    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w82l\" (UniqueName: \"kubernetes.io/projected/1a8c94cf-e6c7-4366-b180-c3db7b37b16e-kube-api-access-5w82l\") on node \"ip-10-0-129-107.ec2.internal\" DevicePath \"\""
Jul 29 20:14:15.460932 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:15.460859    2566 scope.go:117] "RemoveContainer" containerID="bb8ee236a76fe6671e5ee30b44754b7ede3da08b3e21e5e2fedcbfe0d8305ad0"
Jul 29 20:14:16.037396 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:16.037348    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb" event={"ID":"1a8c94cf-e6c7-4366-b180-c3db7b37b16e","Type":"ContainerDied","Data":"bb8ee236a76fe6671e5ee30b44754b7ede3da08b3e21e5e2fedcbfe0d8305ad0"}
Jul 29 20:14:16.037841 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:16.037406    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"
Jul 29 20:14:16.037841 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:16.037475    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb" event={"ID":"1a8c94cf-e6c7-4366-b180-c3db7b37b16e","Type":"ContainerDied","Data":"6436ba9ef3f429a289b4a1e3c0ee68b77eb9c21458cadc26ed84a67265b5b8ed"}
Jul 29 20:14:16.063855 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:16.063823    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"]
Jul 29 20:14:16.067711 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:16.067687    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-601aa9-78687d6f44-d7rqb"]
Jul 29 20:14:17.102324 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:17.102288    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a8c94cf-e6c7-4366-b180-c3db7b37b16e" path="/var/lib/kubelet/pods/1a8c94cf-e6c7-4366-b180-c3db7b37b16e/volumes"
Jul 29 20:14:28.503986 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:28.503957    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65d7444bfc-2rmm5_41a9ba4e-6393-425e-a00b-da3d4494f889/manager/0.log"
Jul 29 20:14:28.977015 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:28.976988    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-667b7bc75f-dg7fb_e4feffa6-de91-412e-b0f1-3f049537c568/copy-manifests/0.log"
Jul 29 20:14:28.989695 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:28.989668    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-667b7bc75f-dg7fb_e4feffa6-de91-412e-b0f1-3f049537c568/manager/0.log"
Jul 29 20:14:31.456127 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:31.456096    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-fzstb_a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf/kuadrant-console-plugin/0.log"
Jul 29 20:14:31.684907 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:31.684878    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2f8n5_6d985cbc-050d-4348-97c8-b90aed917660/manager/0.log"
Jul 29 20:14:32.462388 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:32.462365    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6ffddf6-pc545_b41c7a5b-74ab-449e-8867-e9fa003e722a/kube-auth-proxy/0.log"
Jul 29 20:14:32.790069 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:32.789992    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-zx6xj_cfd9d45c-eab6-4b9f-8adc-8114a772f3ad/payload-pre-processing/0.log"
Jul 29 20:14:33.446311 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:33.446281    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-zcsnb_65016c35-25d7-49a8-9868-8b41aaae058c/main/0.log"
Jul 29 20:14:33.884110 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:33.884029    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-pg4gk_1a82a835-a0d8-45c7-834c-350e95babe7f/main/0.log"
Jul 29 20:14:38.819660 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.819627    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ttn7v/must-gather-8jjp5"]
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.819996    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a8c94cf-e6c7-4366-b180-c3db7b37b16e" containerName="payload-pre-processing"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820007    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a8c94cf-e6c7-4366-b180-c3db7b37b16e" containerName="payload-pre-processing"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820017    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef7edd40-3014-432b-98df-bda6d2f8b700" containerName="main"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820023    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef7edd40-3014-432b-98df-bda6d2f8b700" containerName="main"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820030    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820036    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820048    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51a53f77-9600-44b1-be67-1c556b362c01" containerName="payload-pre-processing"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820054    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="51a53f77-9600-44b1-be67-1c556b362c01" containerName="payload-pre-processing"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820064    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="939a1a9e-0093-4c24-9782-0afafa5a836a" containerName="payload-pre-processing"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820069    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="939a1a9e-0093-4c24-9782-0afafa5a836a" containerName="payload-pre-processing"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820079    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad0207c7-82ae-47cd-ab01-a49f938a1db6" containerName="payload-pre-processing"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820084    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad0207c7-82ae-47cd-ab01-a49f938a1db6" containerName="payload-pre-processing"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820094    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerName="istio-proxy"
Jul 29 20:14:38.820093 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820099    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerName="istio-proxy"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820107    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="833e778b-3ac4-4b31-939b-1694bfd028fe" containerName="payload-processing"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820112    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="833e778b-3ac4-4b31-939b-1694bfd028fe" containerName="payload-processing"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820121    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerName="istio-proxy"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820126    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerName="istio-proxy"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820187    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerName="istio-proxy"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820202    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a8c94cf-e6c7-4366-b180-c3db7b37b16e" containerName="payload-pre-processing"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820211    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="833e778b-3ac4-4b31-939b-1694bfd028fe" containerName="payload-processing"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820219    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="51a53f77-9600-44b1-be67-1c556b362c01" containerName="payload-pre-processing"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820225    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef7edd40-3014-432b-98df-bda6d2f8b700" containerName="main"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820230    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="939a1a9e-0093-4c24-9782-0afafa5a836a" containerName="payload-pre-processing"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820237    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820242    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820249    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad0207c7-82ae-47cd-ab01-a49f938a1db6" containerName="payload-pre-processing"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820310    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820315    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7dd319d-a266-4e3e-bebb-232bc8a9c303" containerName="istio-proxy"
Jul 29 20:14:38.820542 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.820365    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="91b331f4-313d-4c17-b3cc-d4b66ab01bc6" containerName="istio-proxy"
Jul 29 20:14:38.823222 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.823206    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ttn7v/must-gather-8jjp5"
Jul 29 20:14:38.825982 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.825961    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ttn7v\"/\"kube-root-ca.crt\""
Jul 29 20:14:38.826174 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.826141    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ttn7v\"/\"openshift-service-ca.crt\""
Jul 29 20:14:38.826276 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.826193    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ttn7v\"/\"default-dockercfg-4bzhx\""
Jul 29 20:14:38.835725 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.835704    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ttn7v/must-gather-8jjp5"]
Jul 29 20:14:38.982538 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.982508    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4dbb344d-0387-4806-b40a-317026b21d86-must-gather-output\") pod \"must-gather-8jjp5\" (UID: \"4dbb344d-0387-4806-b40a-317026b21d86\") " pod="openshift-must-gather-ttn7v/must-gather-8jjp5"
Jul 29 20:14:38.982703 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:38.982650    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmzr7\" (UniqueName: \"kubernetes.io/projected/4dbb344d-0387-4806-b40a-317026b21d86-kube-api-access-qmzr7\") pod \"must-gather-8jjp5\" (UID: \"4dbb344d-0387-4806-b40a-317026b21d86\") " pod="openshift-must-gather-ttn7v/must-gather-8jjp5"
Jul 29 20:14:39.083819 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:39.083749    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmzr7\" (UniqueName: \"kubernetes.io/projected/4dbb344d-0387-4806-b40a-317026b21d86-kube-api-access-qmzr7\") pod \"must-gather-8jjp5\" (UID: \"4dbb344d-0387-4806-b40a-317026b21d86\") " pod="openshift-must-gather-ttn7v/must-gather-8jjp5"
Jul 29 20:14:39.083819 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:39.083788    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4dbb344d-0387-4806-b40a-317026b21d86-must-gather-output\") pod \"must-gather-8jjp5\" (UID: \"4dbb344d-0387-4806-b40a-317026b21d86\") " pod="openshift-must-gather-ttn7v/must-gather-8jjp5"
Jul 29 20:14:39.084088 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:39.084071    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4dbb344d-0387-4806-b40a-317026b21d86-must-gather-output\") pod \"must-gather-8jjp5\" (UID: \"4dbb344d-0387-4806-b40a-317026b21d86\") " pod="openshift-must-gather-ttn7v/must-gather-8jjp5"
Jul 29 20:14:39.092874 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:39.092855    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmzr7\" (UniqueName: \"kubernetes.io/projected/4dbb344d-0387-4806-b40a-317026b21d86-kube-api-access-qmzr7\") pod \"must-gather-8jjp5\" (UID: \"4dbb344d-0387-4806-b40a-317026b21d86\") " pod="openshift-must-gather-ttn7v/must-gather-8jjp5"
Jul 29 20:14:39.132367 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:39.132343    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ttn7v/must-gather-8jjp5"
Jul 29 20:14:39.261793 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:39.261767    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ttn7v/must-gather-8jjp5"]
Jul 29 20:14:39.264293 ip-10-0-129-107 kubenswrapper[2566]: W0729 20:14:39.264266    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dbb344d_0387_4806_b40a_317026b21d86.slice/crio-6d3b2df0bdcec2443938f8a20b6e6b5783d00919c3f5f9d0c4c0a63797ad9cf0 WatchSource:0}: Error finding container 6d3b2df0bdcec2443938f8a20b6e6b5783d00919c3f5f9d0c4c0a63797ad9cf0: Status 404 returned error can't find the container with id 6d3b2df0bdcec2443938f8a20b6e6b5783d00919c3f5f9d0c4c0a63797ad9cf0
Jul 29 20:14:39.266128 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:39.266111    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 20:14:40.154728 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:40.154703    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ttn7v/must-gather-8jjp5" event={"ID":"4dbb344d-0387-4806-b40a-317026b21d86","Type":"ContainerStarted","Data":"242ec3d30a13f0356887e2854e27a67fc7236a155aa2a68119851ec2e23640e0"}
Jul 29 20:14:40.155012 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:40.154744    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ttn7v/must-gather-8jjp5" event={"ID":"4dbb344d-0387-4806-b40a-317026b21d86","Type":"ContainerStarted","Data":"6d3b2df0bdcec2443938f8a20b6e6b5783d00919c3f5f9d0c4c0a63797ad9cf0"}
Jul 29 20:14:41.163297 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:41.163254    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ttn7v/must-gather-8jjp5" event={"ID":"4dbb344d-0387-4806-b40a-317026b21d86","Type":"ContainerStarted","Data":"be9931550bd73796ce0eb071d672e85ac01b3c7572df594b22fa97075fbe88a9"}
Jul 29 20:14:41.813531 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:41.813495    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ppw48_ea48fa24-04c4-434f-9cb9-eb941d6fff89/global-pull-secret-syncer/0.log"
Jul 29 20:14:41.931564 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:41.931529    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wl5d4_a463d6f0-c349-4a93-8d64-ca9fe04cdc46/konnectivity-agent/0.log"
Jul 29 20:14:41.981996 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:41.981964    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-107.ec2.internal_ccf7f81c6540eccb14cc3f753f1ba29f/haproxy/0.log"
Jul 29 20:14:46.948151 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:46.948120    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-fzstb_a92a83f2-dda8-4fe9-b17a-f3d3fc1e30cf/kuadrant-console-plugin/0.log"
Jul 29 20:14:47.211478 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:47.210043    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2f8n5_6d985cbc-050d-4348-97c8-b90aed917660/manager/0.log"
Jul 29 20:14:49.000208 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.000080    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d03a1bc6-34b8-41b1-96d4-46ebe49c9271/alertmanager/0.log"
Jul 29 20:14:49.021321 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.021295    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d03a1bc6-34b8-41b1-96d4-46ebe49c9271/config-reloader/0.log"
Jul 29 20:14:49.044390 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.044361    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d03a1bc6-34b8-41b1-96d4-46ebe49c9271/kube-rbac-proxy-web/0.log"
Jul 29 20:14:49.082491 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.082443    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d03a1bc6-34b8-41b1-96d4-46ebe49c9271/kube-rbac-proxy/0.log"
Jul 29 20:14:49.107962 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.107935    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d03a1bc6-34b8-41b1-96d4-46ebe49c9271/kube-rbac-proxy-metric/0.log"
Jul 29 20:14:49.141003 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.140975    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d03a1bc6-34b8-41b1-96d4-46ebe49c9271/prom-label-proxy/0.log"
Jul 29 20:14:49.169783 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.169752    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d03a1bc6-34b8-41b1-96d4-46ebe49c9271/init-config-reloader/0.log"
Jul 29 20:14:49.252843 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.252747    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7g8k4_c6cd90e7-355c-4173-8b7f-8febea0aa35a/kube-state-metrics/0.log"
Jul 29 20:14:49.280023 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.279991    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7g8k4_c6cd90e7-355c-4173-8b7f-8febea0aa35a/kube-rbac-proxy-main/0.log"
Jul 29 20:14:49.320471 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.314089    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-7g8k4_c6cd90e7-355c-4173-8b7f-8febea0aa35a/kube-rbac-proxy-self/0.log"
Jul 29 20:14:49.349279 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.349254    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7ffb854bd7-7nf7v_8fc9fe74-1cf6-46cd-a887-ca254d560efb/metrics-server/0.log"
Jul 29 20:14:49.383777 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.383747    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-b66cn_8d642388-3350-4f36-a995-942581144c66/monitoring-plugin/0.log"
Jul 29 20:14:49.594908 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.594876    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t52rl_4aa00892-4616-4831-ac2b-de884a03cf3e/node-exporter/0.log"
Jul 29 20:14:49.615933 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.615901    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t52rl_4aa00892-4616-4831-ac2b-de884a03cf3e/kube-rbac-proxy/0.log"
Jul 29 20:14:49.639335 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.639307    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t52rl_4aa00892-4616-4831-ac2b-de884a03cf3e/init-textfile/0.log"
Jul 29 20:14:49.666208 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.666171    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-955v9_b3b108bf-897c-44f7-80e8-6f9159c40e11/kube-rbac-proxy-main/0.log"
Jul 29 20:14:49.689605 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.689579    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-955v9_b3b108bf-897c-44f7-80e8-6f9159c40e11/kube-rbac-proxy-self/0.log"
Jul 29 20:14:49.711732 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.711705    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-955v9_b3b108bf-897c-44f7-80e8-6f9159c40e11/openshift-state-metrics/0.log"
Jul 29 20:14:49.758954 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.758926    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0c5922cd-43c2-4f12-9005-9e0793c68e9b/prometheus/0.log"
Jul 29 20:14:49.799088 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.799059    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0c5922cd-43c2-4f12-9005-9e0793c68e9b/config-reloader/0.log"
Jul 29 20:14:49.821159 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.821073    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0c5922cd-43c2-4f12-9005-9e0793c68e9b/thanos-sidecar/0.log"
Jul 29 20:14:49.842820 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.842794    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0c5922cd-43c2-4f12-9005-9e0793c68e9b/kube-rbac-proxy-web/0.log"
Jul 29 20:14:49.864744 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.864712    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0c5922cd-43c2-4f12-9005-9e0793c68e9b/kube-rbac-proxy/0.log"
Jul 29 20:14:49.888023 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.887989    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0c5922cd-43c2-4f12-9005-9e0793c68e9b/kube-rbac-proxy-thanos/0.log"
Jul 29 20:14:49.911656 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.911619    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0c5922cd-43c2-4f12-9005-9e0793c68e9b/init-config-reloader/0.log"
Jul 29 20:14:49.955216 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.955182    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-5zqdb_589cf6c9-8947-452f-8566-3b7358ebd3bc/prometheus-operator/0.log"
Jul 29 20:14:49.975420 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:49.975389    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-5zqdb_589cf6c9-8947-452f-8566-3b7358ebd3bc/kube-rbac-proxy/0.log"
Jul 29 20:14:50.001419 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:50.001389    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-5gs8s_8b31b5d7-12fd-4b25-8876-430d3d6cf34b/prometheus-operator-admission-webhook/0.log"
Jul 29 20:14:50.119942 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:50.119856    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fc54567b8-6vgsr_0ea608ef-b17b-4f97-a22e-f83b56183dad/thanos-query/0.log"
Jul 29 20:14:50.139847 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:50.139812    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fc54567b8-6vgsr_0ea608ef-b17b-4f97-a22e-f83b56183dad/kube-rbac-proxy-web/0.log"
Jul 29 20:14:50.161706 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:50.161682    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fc54567b8-6vgsr_0ea608ef-b17b-4f97-a22e-f83b56183dad/kube-rbac-proxy/0.log"
Jul 29 20:14:50.182784 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:50.182746    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fc54567b8-6vgsr_0ea608ef-b17b-4f97-a22e-f83b56183dad/prom-label-proxy/0.log"
Jul 29 20:14:50.203834 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:50.203809    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fc54567b8-6vgsr_0ea608ef-b17b-4f97-a22e-f83b56183dad/kube-rbac-proxy-rules/0.log"
Jul 29 20:14:50.224948 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:50.224918    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5fc54567b8-6vgsr_0ea608ef-b17b-4f97-a22e-f83b56183dad/kube-rbac-proxy-metrics/0.log"
Jul 29 20:14:51.676689 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.676623    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ttn7v/must-gather-8jjp5" podStartSLOduration=12.884497407 podStartE2EDuration="13.676598341s" totalImagesPullingTime="792.100934ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:14:41.183735875 +0000 UTC m=+3446.679497217" watchObservedRunningTime="2026-07-29 20:14:51.676598341 +0000 UTC m=+3457.172359685"
Jul 29 20:14:51.678484 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.678437    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"]
Jul 29 20:14:51.683126 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.683104    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.693163 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.692788    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"]
Jul 29 20:14:51.731892 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.731858    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jkzb\" (UniqueName: \"kubernetes.io/projected/d90016df-c771-4101-ba08-91a7f1807d4b-kube-api-access-7jkzb\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.732091 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.731902    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-podres\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.732091 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.732012    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-sys\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.732091 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.732072    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-proc\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.732259 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.732115    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-lib-modules\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.833110 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.833073    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-sys\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.833311 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.833154    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-proc\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.833311 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.833201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-lib-modules\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.833311 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.833210    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-sys\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.833311 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.833285    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jkzb\" (UniqueName: \"kubernetes.io/projected/d90016df-c771-4101-ba08-91a7f1807d4b-kube-api-access-7jkzb\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.833577 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.833329    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-podres\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.833577 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.833348    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-lib-modules\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.833577 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.833286    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-proc\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.833577 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.833434    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d90016df-c771-4101-ba08-91a7f1807d4b-podres\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.843328 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.843296    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jkzb\" (UniqueName: \"kubernetes.io/projected/d90016df-c771-4101-ba08-91a7f1807d4b-kube-api-access-7jkzb\") pod \"perf-node-gather-daemonset-g6whl\" (UID: \"d90016df-c771-4101-ba08-91a7f1807d4b\") " pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:51.998471 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:51.998414    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:52.188153 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:52.185854    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"]
Jul 29 20:14:52.272616 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:52.272486    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl" event={"ID":"d90016df-c771-4101-ba08-91a7f1807d4b","Type":"ContainerStarted","Data":"cc48eb951bc05bac12cce3d4e5310d4c79ed37b6d25d87c10b8f4b231c6a4080"}
Jul 29 20:14:52.666931 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:52.666859    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-b78748b79-vk5tq_cd2f9792-f42f-49a9-9a9b-c33cea2911fc/console/0.log"
Jul 29 20:14:53.279015 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:53.278973    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl" event={"ID":"d90016df-c771-4101-ba08-91a7f1807d4b","Type":"ContainerStarted","Data":"38d3d7cd881785df93fae9cae542c8411a5b58a6c95a048995ec0492940208e8"}
Jul 29 20:14:53.279394 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:53.279045    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:14:53.298125 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:53.298067    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl" podStartSLOduration=2.2980463589999998 podStartE2EDuration="2.298046359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:14:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:14:53.297820489 +0000 UTC m=+3458.793581831" watchObservedRunningTime="2026-07-29 20:14:53.298046359 +0000 UTC m=+3458.793807702"
Jul 29 20:14:54.383142 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:54.383108    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dbr22_ec82c59a-a7ae-46f8-884d-b3ad641bdb97/dns/0.log"
Jul 29 20:14:54.402197 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:54.402175    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dbr22_ec82c59a-a7ae-46f8-884d-b3ad641bdb97/kube-rbac-proxy/0.log"
Jul 29 20:14:54.514100 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:54.514068    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-99b48_b2377041-6074-4084-9042-655f433c91ed/dns-node-resolver/0.log"
Jul 29 20:14:55.077748 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:55.077716    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7756dbb7c9-wvsz8_b17d923e-a110-4e2a-9c0c-fa0e2d664199/registry/0.log"
Jul 29 20:14:55.127122 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:55.127065    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-86g76_d748fef7-568c-4f8c-9698-6f93e239f253/node-ca/0.log"
Jul 29 20:14:56.259461 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:56.259420    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6ffddf6-pc545_b41c7a5b-74ab-449e-8867-e9fa003e722a/kube-auth-proxy/0.log"
Jul 29 20:14:56.379077 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:56.379050    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-zx6xj_cfd9d45c-eab6-4b9f-8adc-8114a772f3ad/payload-pre-processing/0.log"
Jul 29 20:14:57.199033 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:57.199008    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ctkzl_553abb5a-4c65-47ab-ad96-bfe1e231265c/serve-healthcheck-canary/0.log"
Jul 29 20:14:57.737569 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:57.737545    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bfvhx_e430e745-31c3-48f0-bfaf-ce61e230f95a/kube-rbac-proxy/0.log"
Jul 29 20:14:57.760556 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:57.760532    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bfvhx_e430e745-31c3-48f0-bfaf-ce61e230f95a/exporter/0.log"
Jul 29 20:14:57.793065 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:57.793018    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bfvhx_e430e745-31c3-48f0-bfaf-ce61e230f95a/extractor/0.log"
Jul 29 20:14:59.298054 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:14:59.298025    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ttn7v/perf-node-gather-daemonset-g6whl"
Jul 29 20:15:00.581492 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:00.581384    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65d7444bfc-2rmm5_41a9ba4e-6393-425e-a00b-da3d4494f889/manager/0.log"
Jul 29 20:15:00.844912 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:00.844825    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-667b7bc75f-dg7fb_e4feffa6-de91-412e-b0f1-3f049537c568/manager/0.log"
Jul 29 20:15:00.870507 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:00.870433    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-667b7bc75f-dg7fb_e4feffa6-de91-412e-b0f1-3f049537c568/copy-manifests/0.log"
Jul 29 20:15:02.291123 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:02.291096    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-848988bfdc-zqhxc_5dc699ad-8dcc-4d71-bff6-a9b356a281b3/manager/0.log"
Jul 29 20:15:02.430177 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:02.430143    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6ffddf6-pc545_b41c7a5b-74ab-449e-8867-e9fa003e722a/kube-auth-proxy/0.log"
Jul 29 20:15:02.518931 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:02.518897    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-zx6xj_cfd9d45c-eab6-4b9f-8adc-8114a772f3ad/payload-pre-processing/0.log"
Jul 29 20:15:04.091807 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:04.091730    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6ffddf6-pc545_b41c7a5b-74ab-449e-8867-e9fa003e722a/kube-auth-proxy/0.log"
Jul 29 20:15:04.180134 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:04.180104    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-zx6xj_cfd9d45c-eab6-4b9f-8adc-8114a772f3ad/payload-pre-processing/0.log"
Jul 29 20:15:08.643560 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:08.643531    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9rhjr_1dcfa88a-bacc-40bc-bb01-fb73b300931d/kube-multus/0.log"
Jul 29 20:15:08.666101 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:08.666075    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gczs9_a5e3afa4-7206-4362-9c74-32b04f05d4d6/kube-multus-additional-cni-plugins/0.log"
Jul 29 20:15:08.686042 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:08.686018    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gczs9_a5e3afa4-7206-4362-9c74-32b04f05d4d6/egress-router-binary-copy/0.log"
Jul 29 20:15:08.715837 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:08.715816    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gczs9_a5e3afa4-7206-4362-9c74-32b04f05d4d6/cni-plugins/0.log"
Jul 29 20:15:08.735048 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:08.734998    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gczs9_a5e3afa4-7206-4362-9c74-32b04f05d4d6/bond-cni-plugin/0.log"
Jul 29 20:15:08.756660 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:08.756630    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gczs9_a5e3afa4-7206-4362-9c74-32b04f05d4d6/routeoverride-cni/0.log"
Jul 29 20:15:08.776198 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:08.776169    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gczs9_a5e3afa4-7206-4362-9c74-32b04f05d4d6/whereabouts-cni-bincopy/0.log"
Jul 29 20:15:08.801189 ip-10-0-129-107 kubenswrapper[2566]: I0729 20:15:08.801159    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gczs9_a5e3afa4-7206-4362-9c74-32b04f05d4d6/whereabouts-cni/0.log"