Sep 08 13:58:28.331548 ip-10-0-133-79 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 13:58:28.701512 ip-10-0-133-79 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 13:58:28.701512 ip-10-0-133-79 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 13:58:28.701512 ip-10-0-133-79 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 13:58:28.701512 ip-10-0-133-79 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 13:58:28.701512 ip-10-0-133-79 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 13:58:28.703771 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.703681    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 13:58:28.707814 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707798    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:58:28.707814 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707814    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707818    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707821    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707824    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707828    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707832    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707835    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707839    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707841    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707844    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707847    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707849    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707852    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707855    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707858    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707860    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707863    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707866    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707872    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707875    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:58:28.707880 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707878    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707880    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707884    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707887    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707890    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707892    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707895    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707898    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707902    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707904    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707907    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707910    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707912    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707915    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707918    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707921    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707925    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707929    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707932    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:58:28.708376 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707934    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707937    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707939    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707942    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707944    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707947    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707949    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707952    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707954    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707957    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707960    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707962    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707965    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707967    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707969    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707973    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707976    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707979    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707981    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707984    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:58:28.708855 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707986    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707990    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707993    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707995    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.707998    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708000    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708003    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708006    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708009    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708012    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708014    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708017    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708019    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708022    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708024    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708027    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708029    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708032    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708034    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708036    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:58:28.709352 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708039    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708041    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708044    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708046    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708049    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708052    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708054    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708059    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708063    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708066    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708068    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708070    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708073    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708075    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708078    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708080    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708084    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708087    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708091    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:58:28.709821 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708801    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708807    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708811    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708813    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708817    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708820    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708823    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708825    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708828    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708831    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708833    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708836    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708839    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708841    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708844    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708847    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708850    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708852    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708855    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708858    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:58:28.710304 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708861    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708864    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708867    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708870    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708873    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708877    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708881    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708884    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708887    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708889    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708892    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708895    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708898    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708902    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708904    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708907    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708909    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708912    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708914    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:58:28.710806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708917    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708920    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708923    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708927    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708930    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708933    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708935    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708938    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708940    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708943    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708945    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708948    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708950    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708955    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708958    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708960    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708963    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708966    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708968    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708971    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:58:28.711263 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708973    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708976    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708978    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708981    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708983    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708986    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708989    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708992    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708994    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708997    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.708999    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709002    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709004    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709007    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709009    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709012    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709015    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709018    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709020    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709023    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:58:28.711762 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709025    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709028    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709030    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709034    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709038    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709041    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709044    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709047    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709050    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709052    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709055    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709057    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709060    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709063    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709065    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709068    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709070    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709073    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709075    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:58:28.712256 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709078    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709464    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709469    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709472    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709475    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709478    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709482    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709486    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709489    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709492    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709495    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709498    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709501    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709503    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709506    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709509    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709513    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709516    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709520    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:58:28.712744 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709524    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709527    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709529    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709532    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709534    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709537    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709539    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709542    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709544    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709547    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709549    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709552    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709555    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709557    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709559    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709564    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709567    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709570    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709572    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:58:28.713208 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709575    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709578    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709581    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709584    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709587    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709589    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709592    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709595    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709598    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709600    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709604    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709607    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709610    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709613    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709616    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709619    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709622    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709625    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709628    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709630    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:58:28.713772 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709633    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709635    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709638    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709640    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709643    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709646    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709649    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709651    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709654    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709656    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709659    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709661    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709664    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709666    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709669    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709671    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709674    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709676    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709679    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709681    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:58:28.714319 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709684    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709687    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709690    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709693    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709697    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709700    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709702    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709705    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709707    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709730    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709733    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709736    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709739    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709742    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709745    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709747    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709750    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709752    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709755    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709758    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:58:28.714806 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709761    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.709765    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709843    2564 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709892    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709900    2564 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709904    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709908    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709911    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709915    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709919    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709923    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709926    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709930    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709933    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709937    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709939    2564 flags.go:64] FLAG: --cgroup-root=""
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709942    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709945    2564 flags.go:64] FLAG: --client-ca-file=""
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709948    2564 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709951    2564 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709956    2564 flags.go:64] FLAG: --cluster-domain=""
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709959    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709962    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709965    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 13:58:28.715302 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709968    2564 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709972    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709975    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709978    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709981    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709984    2564 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709989    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709992    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709995    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.709998    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710002    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710005    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710008    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710011    2564 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710014    2564 flags.go:64] FLAG: --enable-server="true"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710017    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710021    2564 flags.go:64] FLAG: --event-burst="100"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710024    2564 flags.go:64] FLAG: --event-qps="50"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710026    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710029    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710032    2564 flags.go:64] FLAG: --eviction-hard=""
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710036    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710039    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710042    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710045    2564 flags.go:64] FLAG: --eviction-soft=""
Sep 08 13:58:28.715940 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710047    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710050    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710053    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710056    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710059    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710062    2564 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710065    2564 flags.go:64] FLAG: --feature-gates=""
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710069    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710071    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710074    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710077    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710080    2564 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710083    2564 flags.go:64] FLAG: --help="false"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710087    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-133-79.ec2.internal"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710090    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710093    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710096    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710099    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710103    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710105    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710108    2564 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710111    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710113    2564 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 13:58:28.716557 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710116    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710120    2564 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710123    2564 flags.go:64] FLAG: --kube-reserved=""
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710126    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710128    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710131    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710134    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710137    2564 flags.go:64] FLAG: --lock-file=""
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710139    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710142    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710145    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710150    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710153    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710156    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710158    2564 flags.go:64] FLAG: --logging-format="text"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710161    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710164    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710166    2564 flags.go:64] FLAG: --manifest-url=""
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710169    2564 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710173    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710176    2564 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710180    2564 flags.go:64] FLAG: --max-pods="110"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710184    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710186    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710189    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 13:58:28.717130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710192    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710195    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710198    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710201    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710208    2564 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710211    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710214    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710217    2564 flags.go:64] FLAG: --pod-cidr=""
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710220    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710226    2564 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710229    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710232    2564 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710235    2564 flags.go:64] FLAG: --port="10250"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710238    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710241    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e99ff5fe8b24e9c1"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710245    2564 flags.go:64] FLAG: --qos-reserved=""
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710248    2564 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710250    2564 flags.go:64] FLAG: --register-node="true"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710253    2564 flags.go:64] FLAG: --register-with-taints=""
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710262    2564 flags.go:64] FLAG: --registry-burst="10"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710265    2564 flags.go:64] FLAG: --registry-qps="5"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710267    2564 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710285    2564 flags.go:64] FLAG: --reserved-memory=""
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710289    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710292    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 13:58:28.717766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710295    2564 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710298    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710301    2564 flags.go:64] FLAG: --runonce="false"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710304    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710308    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710312    2564 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710315    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710318    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710321    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710325    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710328    2564 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710331    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710334    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710337    2564 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710340    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710343    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710346    2564 flags.go:64] FLAG: --system-cgroups=""
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710349    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710354    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710357    2564 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710360    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710365    2564 flags.go:64] FLAG: --tls-min-version=""
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710367    2564 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710370    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710373    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 13:58:28.718381 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710376    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710378    2564 flags.go:64] FLAG: --v="2"
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710382    2564 flags.go:64] FLAG: --version="false"
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710386    2564 flags.go:64] FLAG: --vmodule=""
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710390    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.710393    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710509    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710513    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710516    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710519    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710522    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710525    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710528    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710530    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710533    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710535    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710539    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710542    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710545    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710547    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710550    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:58:28.719134 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710552    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710555    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710557    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710560    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710562    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710565    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710569    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710572    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710575    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710578    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710582    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710585    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710588    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710591    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710593    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710596    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710599    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710616    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:58:28.719862 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710620    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710622    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710625    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710628    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710631    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710633    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710636    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710639    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710642    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710645    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710647    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710651    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710653    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710656    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710658    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710661    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710665    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710667    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710670    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710673    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:58:28.720335 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710675    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710678    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710680    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710683    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710685    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710688    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710690    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710693    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710695    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710698    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710700    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710703    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710705    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710708    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710710    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710713    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710715    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710718    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710720    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710723    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710725    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:58:28.720818 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710728    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710730    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710733    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710736    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710739    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710741    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710744    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710747    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710750    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710753    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710755    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710758    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710760    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710762    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710765    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710768    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710770    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710773    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710775    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710778    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:58:28.721426 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710780    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710783    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710786    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710788    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.710792    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.711372    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.720228    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.720249    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.720269    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.720296    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720362    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720368    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720372    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720375    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:58:28.721904 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720377    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720380    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720383    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720385    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720388    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720392    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720397    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720401    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720405    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720410    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720412    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720416    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720418    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720421    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720423    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720426    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720429    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720431    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720435    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720439    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:58:28.722255 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720443    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720446    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720449    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720452    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720456    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720460    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720463    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720465    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720468    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720471    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720475    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720479    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720484    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720488    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720492    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720495    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720498    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720500    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720503    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720505    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:58:28.722754 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720508    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720510    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720513    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720515    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720518    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720521    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720523    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720526    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720530    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720533    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720538    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720541    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720544    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720547    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720550    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720553    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720557    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720563    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720567    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:58:28.723289 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720572    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720575    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720577    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720580    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720583    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720585    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720588    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720590    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720593    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720595    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720598    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720600    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720603    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720605    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720607    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720610    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720613    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720616    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720618    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720621    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:58:28.723757 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720623    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720626    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720629    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720631    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720634    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720637    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720642    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720646    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720650    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720654    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720657    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720660    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720663    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720665    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720668    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:58:28.724249 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720671    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.720676    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720801    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720810    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720815    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720819    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720823    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720825    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720829    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720831    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720834    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720837    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720839    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720842    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720845    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:58:28.724635 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720848    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720852    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720856    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720858    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720861    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720864    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720867    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720871    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720873    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720876    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720879    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720882    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720886    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720891    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720895    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720900    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720904    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720907    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720910    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:58:28.725002 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720912    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720915    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720918    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720920    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720923    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720925    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720928    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720930    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720933    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720936    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720938    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720941    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720943    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720947    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720949    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720952    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720954    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720957    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720959    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:58:28.725482 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720963    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720968    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720973    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720977    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720980    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720983    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720985    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720989    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720991    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720994    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720997    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.720999    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721002    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721004    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721007    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721009    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721011    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721014    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721016    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721019    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:58:28.725931 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721021    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721024    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721026    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721029    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721031    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721034    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721036    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721040    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721043    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721047    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721051    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721056    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721060    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721063    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721066    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721070    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721073    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721076    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721079    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:58:28.726434 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721081    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721084    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721086    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721089    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721092    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721094    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721097    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721099    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:28.721101    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.721106    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.721236    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.724708    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.725708    2564 server.go:1019] "Starting client certificate rotation"
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.725808    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 13:58:28.726889 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.725850    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 13:58:28.727953 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.727939    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 13:58:28.729182 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.729162    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 13:58:28.736912 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.736896    2564 log.go:25] "Validated CRI v1 runtime API"
Sep 08 13:58:28.739544 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.739527    2564 log.go:25] "Validated CRI v1 image API"
Sep 08 13:58:28.740359 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.740344    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 13:58:28.744995 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.744975    2564 fs.go:135] Filesystem UUIDs: map[5e401831-b7a9-499a-be6f-4d820fb7b175:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ce3652bd-225f-460e-8578-236267b7781c:/dev/nvme0n1p3]
Sep 08 13:58:28.745072 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.744993    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 13:58:28.751447 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.751339    2564 manager.go:216] Machine: {Timestamp:2026-09-08 13:58:28.749493512 +0000 UTC m=+0.305054051 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103222 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec256f94545152fe91e550f663e734a2 SystemUUID:ec256f94-5451-52fe-91e5-50f663e734a2 BootID:7cec1117-6086-4fe2-9f8c-ea36eb1e0876 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4c:fc:48:83:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4c:fc:48:83:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:76:38:0a:e0:aa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 13:58:28.751447 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.751443    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 13:58:28.751555 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.751530    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 13:58:28.753406 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.753381    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 13:58:28.753627 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.753409    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-79.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 13:58:28.754605 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.754594    2564 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 13:58:28.754636 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.754609    2564 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 13:58:28.754636 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.754618    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 13:58:28.755215 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.755205    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 13:58:28.755382 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.755368    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 13:58:28.756500 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.756489    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 13:58:28.759477 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.759462    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 13:58:28.759586 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.759577    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 13:58:28.759630 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.759606    2564 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 13:58:28.759630 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.759623    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 13:58:28.759711 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.759634    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 13:58:28.759711 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.759646    2564 kubelet.go:398] "Adding apiserver pod source"
Sep 08 13:58:28.759711 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.759655    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 13:58:28.760598 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.760584    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 13:58:28.760669 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.760603    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 13:58:28.762660 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.762643    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 13:58:28.764909 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.764894    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 13:58:28.764978 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.764925    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 13:58:28.765080 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765071    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 13:58:28.765111 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765085    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 13:58:28.765111 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765091    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 13:58:28.765111 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765097    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 13:58:28.765111 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765105    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 13:58:28.765111 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765110    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 13:58:28.765534 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765525    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 13:58:28.765534 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765534    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 13:58:28.765600 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765541    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 13:58:28.765600 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765546    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 13:58:28.765600 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765554    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 13:58:28.765600 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.765561    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 13:58:28.766146 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.766135    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 13:58:28.766146 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.766145    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 13:58:28.768682 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.768662    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-79.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 13:58:28.768749 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.768660    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-79.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 13:58:28.768749 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.768656    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 13:58:28.769832 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.769816    2564 server.go:1268] "Started kubelet"
Sep 08 13:58:28.769931 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.769881    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 13:58:28.770145 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.770004    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 13:58:28.770290 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.770256    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 13:58:28.770781 ip-10-0-133-79 systemd[1]: Started Kubernetes Kubelet.
Sep 08 13:58:28.771195 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.771111    2564 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 13:58:28.773891 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.773859    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 13:58:28.778774 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.778758    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 13:58:28.779067 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.779051    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 13:58:28.780054 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.780032    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 13:58:28.780054 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.780052    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 13:58:28.782318 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.779414    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:28.782434 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.782402    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 13:58:28.782617 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.782603    2564 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 13:58:28.782617 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.782615    2564 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 13:58:28.783086 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.783058    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 13:58:28.783520 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.783494    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fmt9l"
Sep 08 13:58:28.783593 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.782550    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-79.ec2.internal.18d35d29417a3d5b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-79.ec2.internal,UID:ip-10-0-133-79.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-79.ec2.internal,},FirstTimestamp:2026-09-08 13:58:28.769791323 +0000 UTC m=+0.325351861,LastTimestamp:2026-09-08 13:58:28.769791323 +0000 UTC m=+0.325351861,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-79.ec2.internal,}"
Sep 08 13:58:28.783593 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.783588    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 13:58:28.783698 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.783602    2564 factory.go:55] Registering systemd factory
Sep 08 13:58:28.783698 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.783610    2564 factory.go:223] Registration of the systemd container factory successfully
Sep 08 13:58:28.783852 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.783839    2564 factory.go:153] Registering CRI-O factory
Sep 08 13:58:28.783904 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.783856    2564 factory.go:223] Registration of the crio container factory successfully
Sep 08 13:58:28.783904 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.783877    2564 factory.go:103] Registering Raw factory
Sep 08 13:58:28.783904 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.783891    2564 manager.go:1196] Started watching for new ooms in manager
Sep 08 13:58:28.784327 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.784314    2564 manager.go:349] Starting recovery of all containers
Sep 08 13:58:28.791458 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.791435    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fmt9l"
Sep 08 13:58:28.796049 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.795123    2564 manager.go:354] Recovery completed
Sep 08 13:58:28.796112 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.796043    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-79.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 08 13:58:28.796112 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.796096    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 08 13:58:28.796612 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.796579    2564 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 08 13:58:28.799706 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.799693    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:58:28.801645 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.801586    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:58:28.801645 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.801622    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:58:28.801645 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.801636    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:58:28.802231 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.802213    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 13:58:28.802231 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.802227    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:58:28.802231 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.802230    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 13:58:28.802380 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.802245    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 13:58:28.802905 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.802892    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:58:28.802975 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.802916    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:58:28.802975 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.802925    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:58:28.803540 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.803528    2564 policy_none.go:49] "None policy: Start"
Sep 08 13:58:28.803583 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.803547    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 13:58:28.803583 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.803560    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 13:58:28.804725 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.804712    2564 policy_none.go:47] "Start"
Sep 08 13:58:28.846088 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.845902    2564 manager.go:337] "Starting Device Plugin manager"
Sep 08 13:58:28.861045 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.846121    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 13:58:28.861045 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.846136    2564 server.go:87] "Starting device plugin registration server"
Sep 08 13:58:28.861045 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.846478    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 13:58:28.861045 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.846489    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 13:58:28.861045 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.846655    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 13:58:28.861045 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.846738    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 13:58:28.861045 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.846747    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 13:58:28.861045 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.847265    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 13:58:28.861045 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.847332    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:28.890180 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.890148    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 13:58:28.891480 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.891467    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 13:58:28.891546 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.891522    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 13:58:28.891546 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.891546    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 13:58:28.891632 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.891581    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 08 13:58:28.894348 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.894333    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:58:28.947660 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.947624    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:58:28.948648 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.948623    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:58:28.948740 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.948656    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:58:28.948740 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.948671    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:58:28.948740 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.948694    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-79.ec2.internal"
Sep 08 13:58:28.961792 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.961744    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-79.ec2.internal"
Sep 08 13:58:28.961792 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.961772    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-79.ec2.internal\": node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:28.981352 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:28.981322    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:28.992616 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.992592    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-79.ec2.internal"]
Sep 08 13:58:28.992671 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.992661    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:58:28.993544 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.993529    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:58:28.993642 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.993562    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:58:28.993642 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.993576    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:58:28.995635 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.995620    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:58:28.995741 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.995727    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:28.995785 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.995754    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:58:28.996312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.996301    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:58:28.996384 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.996324    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:58:28.996384 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.996339    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:58:28.996384 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.996338    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:58:28.996384 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.996373    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:58:28.996384 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.996386    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:58:28.998315 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.998301    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:28.998357 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.998329    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:58:28.998980 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.998959    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:58:28.999062 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.998985    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:58:28.999062 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:28.998998    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:58:29.013809 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.013788    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-79.ec2.internal\" not found" node="ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.018200 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.018184    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-79.ec2.internal\" not found" node="ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.081839 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.081812    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:29.085083 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.085068    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c409c5eae75092b9e98660579ed5eaf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal\" (UID: \"6c409c5eae75092b9e98660579ed5eaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.085155 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.085091    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24342c83cc108d3ad0b5e1eda2e8279a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-79.ec2.internal\" (UID: \"24342c83cc108d3ad0b5e1eda2e8279a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.085155 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.085109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6c409c5eae75092b9e98660579ed5eaf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal\" (UID: \"6c409c5eae75092b9e98660579ed5eaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.182856 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.182814    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:29.186025 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.186006    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6c409c5eae75092b9e98660579ed5eaf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal\" (UID: \"6c409c5eae75092b9e98660579ed5eaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.186085 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.186033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c409c5eae75092b9e98660579ed5eaf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal\" (UID: \"6c409c5eae75092b9e98660579ed5eaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.186085 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.186051    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24342c83cc108d3ad0b5e1eda2e8279a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-79.ec2.internal\" (UID: \"24342c83cc108d3ad0b5e1eda2e8279a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.186085 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.186077    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24342c83cc108d3ad0b5e1eda2e8279a-config\") pod \"kube-apiserver-proxy-ip-10-0-133-79.ec2.internal\" (UID: \"24342c83cc108d3ad0b5e1eda2e8279a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.186210 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.186099    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6c409c5eae75092b9e98660579ed5eaf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal\" (UID: \"6c409c5eae75092b9e98660579ed5eaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.186210 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.186125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c409c5eae75092b9e98660579ed5eaf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal\" (UID: \"6c409c5eae75092b9e98660579ed5eaf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.283760 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.283680    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:29.315980 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.315956    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.320933 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.320456    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:29.384150 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.384121    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:29.484767 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.484746    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:29.585434 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.585372    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:29.686014 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.685983    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:29.702844 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.702820    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:58:29.725962 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.725938    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 13:58:29.726081 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.726063    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 13:58:29.726121 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.726102    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 13:58:29.786502 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.786474    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:29.794910 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.794871    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 13:53:29 +0000 UTC" deadline="2028-02-07 05:07:35.554593501 +0000 UTC"
Sep 08 13:58:29.794910 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.794903    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12399h9m5.759694541s"
Sep 08 13:58:29.887493 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.887466    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:29.987582 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:29.987558    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:29.994547 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:29.994520    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24342c83cc108d3ad0b5e1eda2e8279a.slice/crio-2fd0c04761f0896fd6901bb96beecc4d03cadd1525195f5e90b88627b194eb56 WatchSource:0}: Error finding container 2fd0c04761f0896fd6901bb96beecc4d03cadd1525195f5e90b88627b194eb56: Status 404 returned error can't find the container with id 2fd0c04761f0896fd6901bb96beecc4d03cadd1525195f5e90b88627b194eb56
Sep 08 13:58:29.998790 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:29.998776    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 13:58:30.087791 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:30.087764    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:30.188417 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:30.188355    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-79.ec2.internal\" not found"
Sep 08 13:58:30.253714 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:30.253683    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:58:31.328491 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.328473    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:58:31.340422 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.340401    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:31.352140 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.352119    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 13:58:31.353011 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.352998    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-79.ec2.internal"
Sep 08 13:58:31.362124 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.362106    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 13:58:31.821754 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.821718    2564 apiserver.go:52] "Watching apiserver"
Sep 08 13:58:31.830230 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.830207    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 13:58:31.831410 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.831386    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-bqjtj","openshift-dns/node-resolver-qgvf6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal","openshift-multus/multus-additional-cni-plugins-nhsp9","openshift-multus/multus-gvwgp","kube-system/konnectivity-agent-g8grv","openshift-image-registry/node-ca-47jpr","openshift-multus/network-metrics-daemon-zkd7c","openshift-network-diagnostics/network-check-target-5mbt5","openshift-network-operator/iptables-alerter-kdhc6","openshift-ovn-kubernetes/ovnkube-node-zsjl2","kube-system/global-pull-secret-syncer-lfxpp","kube-system/kube-apiserver-proxy-ip-10-0-133-79.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"]
Sep 08 13:58:31.834148 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.834128    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-47jpr"
Sep 08 13:58:31.836310 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.836290    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.837425 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.837114    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z68pw\""
Sep 08 13:58:31.837425 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.837211    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 13:58:31.837425 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.837120    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 13:58:31.837425 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.837364    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 13:58:31.838377 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.838360    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 13:58:31.838480 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.838465    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 13:58:31.838541 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.838495    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.838652 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.838635    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ggthf\""
Sep 08 13:58:31.838711 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.838652    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 13:58:31.839744 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.839587    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 13:58:31.839744 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.839658    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 13:58:31.839744 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.839668    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 13:58:31.840908 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.840888    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 13:58:31.841012 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.840976    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 13:58:31.841118 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.841101    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 13:58:31.841535 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.841515    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p42l8\""
Sep 08 13:58:31.841827 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.841808    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.843468 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.843380    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 13:58:31.844130 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.844113    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.845069 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.845051    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 13:58:31.845169 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.845146    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 13:58:31.845310 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.845287    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 13:58:31.845660 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.845639    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 13:58:31.845844 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.845828    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zcl99\""
Sep 08 13:58:31.846146 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.846133    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 13:58:31.846431 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.846411    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rjtfm\""
Sep 08 13:58:31.846646 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.846628    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:31.847094 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.847075    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 13:58:31.850796 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.850756    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:31.850948 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:31.850921    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:31.851449 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.851432    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 13:58:31.851537 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.851432    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 13:58:31.851629 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.851540    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qhlqm\""
Sep 08 13:58:31.853155 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853136    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:31.853242 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:31.853213    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:31.853651 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853385    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-cni-netd\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.853651 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853412    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5zjb\" (UniqueName: \"kubernetes.io/projected/05477ef7-e414-40b8-9d2d-b7da68fd264f-kube-api-access-l5zjb\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.853651 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853428    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-var-lib-kubelet\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.853651 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853442    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-daemon-config\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.853651 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853460    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-etc-kubernetes\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.853651 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853484    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-systemd-units\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.853651 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853583    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-run-ovn\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.853651 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853621    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05477ef7-e414-40b8-9d2d-b7da68fd264f-system-cni-dir\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.853651 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853645    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05477ef7-e414-40b8-9d2d-b7da68fd264f-cnibin\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853662    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-socket-dir-parent\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853675    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-run-netns\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853689    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-kubelet\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853712    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-slash\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853734    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-var-lib-openvswitch\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853752    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-run-ovn-kubernetes\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853776    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853801    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/05477ef7-e414-40b8-9d2d-b7da68fd264f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853826    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-cnibin\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-var-lib-cni-multus\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853911    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-run-systemd\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853937    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56gsj\" (UniqueName: \"kubernetes.io/projected/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-kube-api-access-56gsj\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853961    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-etc-selinux\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853984    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crmxx\" (UniqueName: \"kubernetes.io/projected/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-kube-api-access-crmxx\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.853998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/05477ef7-e414-40b8-9d2d-b7da68fd264f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.854177 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-cni-dir\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854037    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7869bcf-f35e-43bf-baa4-04fe6606396a-cni-binary-copy\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-run-k8s-cni-cncf-io\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854069    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da52533e-1455-4ff1-8942-6cf7633a85a1-serviceca\") pod \"node-ca-47jpr\" (UID: \"da52533e-1455-4ff1-8942-6cf7633a85a1\") " pod="openshift-image-registry/node-ca-47jpr"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854089    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncvf9\" (UniqueName: \"kubernetes.io/projected/da52533e-1455-4ff1-8942-6cf7633a85a1-kube-api-access-ncvf9\") pod \"node-ca-47jpr\" (UID: \"da52533e-1455-4ff1-8942-6cf7633a85a1\") " pod="openshift-image-registry/node-ca-47jpr"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854112    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-run-netns\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854134    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854150    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-socket-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854163    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-registration-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854204    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-device-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854226    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-node-log\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-sys-fs\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05477ef7-e414-40b8-9d2d-b7da68fd264f-cni-binary-copy\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854294    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05477ef7-e414-40b8-9d2d-b7da68fd264f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-var-lib-cni-bin\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-hostroot\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.854987 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854352    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzgwq\" (UniqueName: \"kubernetes.io/projected/f7869bcf-f35e-43bf-baa4-04fe6606396a-kube-api-access-kzgwq\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854373    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da52533e-1455-4ff1-8942-6cf7633a85a1-host\") pod \"node-ca-47jpr\" (UID: \"da52533e-1455-4ff1-8942-6cf7633a85a1\") " pod="openshift-image-registry/node-ca-47jpr"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854397    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-ovnkube-config\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854425    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-env-overrides\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854459    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-ovn-node-metrics-cert\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-ovnkube-script-lib\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854535    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-run-multus-certs\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854559    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-run-openvswitch\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854581    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-log-socket\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854604    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-cni-bin\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854655    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-system-cni-dir\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854682    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-conf-dir\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.855699 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.854707    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-etc-openvswitch\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.856803 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.856780    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 13:58:31.857553 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.857399    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qgvf6"
Sep 08 13:58:31.859829 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.859812    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kdhc6"
Sep 08 13:58:31.860275 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.860260    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 13:58:31.860567 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.860550    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s44md\""
Sep 08 13:58:31.860684 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.860647    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 13:58:31.862202 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.862185    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.864799 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.864613    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:31.864799 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:31.864675    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:31.867003 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.866814    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 13:58:31.867003 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.866854    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:58:31.867003 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.866896    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mjvrl\""
Sep 08 13:58:31.867003 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.866962    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 13:58:31.867246 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.867027    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:58:31.867246 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.867089    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-k2ww8\""
Sep 08 13:58:31.867246 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.867131    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 13:58:31.890434 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.890415    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:58:31.893124 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.893102    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w6q59"
Sep 08 13:58:31.902777 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.902746    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w6q59"
Sep 08 13:58:31.945170 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.945147    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 13:58:31.955103 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.954891    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7869bcf-f35e-43bf-baa4-04fe6606396a-cni-binary-copy\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.955103 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.954930    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:31.955103 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.954968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-sysconfig\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.955103 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.954990    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-kubernetes\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.955103 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955015    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-hostroot\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.955103 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955039    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-run-netns\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.955103 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955081    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-hostroot\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.955103 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955078    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-socket-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955131    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-device-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955086    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-run-netns\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-var-lib-kubelet\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955192    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srqg2\" (UniqueName: \"kubernetes.io/projected/6f5f9f31-0c01-410a-ba00-e9e2a20272be-kube-api-access-srqg2\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955218    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55cm4\" (UniqueName: \"kubernetes.io/projected/dbfe671c-f26a-484b-8e4a-d634b1731f98-kube-api-access-55cm4\") pod \"node-resolver-qgvf6\" (UID: \"dbfe671c-f26a-484b-8e4a-d634b1731f98\") " pod="openshift-dns/node-resolver-qgvf6"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955222    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-device-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-socket-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955243    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fad7502f-aea7-4f92-9cd2-7bc9d4df2a62-iptables-alerter-script\") pod \"iptables-alerter-kdhc6\" (UID: \"fad7502f-aea7-4f92-9cd2-7bc9d4df2a62\") " pod="openshift-network-operator/iptables-alerter-kdhc6"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955281    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-sys\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-host\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955380    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da52533e-1455-4ff1-8942-6cf7633a85a1-host\") pod \"node-ca-47jpr\" (UID: \"da52533e-1455-4ff1-8942-6cf7633a85a1\") " pod="openshift-image-registry/node-ca-47jpr"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955428    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da52533e-1455-4ff1-8942-6cf7633a85a1-host\") pod \"node-ca-47jpr\" (UID: \"da52533e-1455-4ff1-8942-6cf7633a85a1\") " pod="openshift-image-registry/node-ca-47jpr"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-ovnkube-config\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955474    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-ovnkube-script-lib\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955501    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-sysctl-d\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955524    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-sysctl-conf\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.955579 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955567    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2b891b6-0ff7-4267-b030-ee95f25a0727-dbus\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955602    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-run-openvswitch\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-conf-dir\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955656    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkbd4\" (UniqueName: \"kubernetes.io/projected/fad7502f-aea7-4f92-9cd2-7bc9d4df2a62-kube-api-access-hkbd4\") pod \"iptables-alerter-kdhc6\" (UID: \"fad7502f-aea7-4f92-9cd2-7bc9d4df2a62\") " pod="openshift-network-operator/iptables-alerter-kdhc6"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955681    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2b891b6-0ff7-4267-b030-ee95f25a0727-kubelet-config\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955708    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-cni-netd\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955710    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-run-openvswitch\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955732    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-conf-dir\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955735    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5zjb\" (UniqueName: \"kubernetes.io/projected/05477ef7-e414-40b8-9d2d-b7da68fd264f-kube-api-access-l5zjb\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-daemon-config\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955809    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-tuned\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955836    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e10f70fd-1430-4819-b75b-a236e679fb6f-agent-certs\") pod \"konnectivity-agent-g8grv\" (UID: \"e10f70fd-1430-4819-b75b-a236e679fb6f\") " pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955879    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-socket-dir-parent\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955906    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-run-netns\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955929    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-slash\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955969    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.955996    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-cnibin\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.956461 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-lib-modules\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956028    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-ovnkube-config\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956030    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-cni-netd\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956047    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-run-multus-certs\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-etc-selinux\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crmxx\" (UniqueName: \"kubernetes.io/projected/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-kube-api-access-crmxx\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956146    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-cnibin\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956154    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-socket-dir-parent\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-run-k8s-cni-cncf-io\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956203    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-run-multus-certs\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956210    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dbfe671c-f26a-484b-8e4a-d634b1731f98-hosts-file\") pod \"node-resolver-qgvf6\" (UID: \"dbfe671c-f26a-484b-8e4a-d634b1731f98\") " pod="openshift-dns/node-resolver-qgvf6"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05477ef7-e414-40b8-9d2d-b7da68fd264f-cni-binary-copy\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956270    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-slash\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzgwq\" (UniqueName: \"kubernetes.io/projected/f7869bcf-f35e-43bf-baa4-04fe6606396a-kube-api-access-kzgwq\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da52533e-1455-4ff1-8942-6cf7633a85a1-serviceca\") pod \"node-ca-47jpr\" (UID: \"da52533e-1455-4ff1-8942-6cf7633a85a1\") " pod="openshift-image-registry/node-ca-47jpr"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncvf9\" (UniqueName: \"kubernetes.io/projected/da52533e-1455-4ff1-8942-6cf7633a85a1-kube-api-access-ncvf9\") pod \"node-ca-47jpr\" (UID: \"da52533e-1455-4ff1-8942-6cf7633a85a1\") " pod="openshift-image-registry/node-ca-47jpr"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956334    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-etc-selinux\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.957312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956330    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-run-k8s-cni-cncf-io\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956344    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956368    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956373    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-run-netns\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-registration-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956418    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-node-log\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956426    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-registration-dir\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956442    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-sys-fs\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956464    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956466    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05477ef7-e414-40b8-9d2d-b7da68fd264f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956518    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-var-lib-cni-bin\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956566    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-sys-fs\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956591    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05477ef7-e414-40b8-9d2d-b7da68fd264f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956836    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-var-lib-cni-bin\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956887    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-env-overrides\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956914    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-ovn-node-metrics-cert\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbnm7\" (UniqueName: \"kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7\") pod \"network-check-target-5mbt5\" (UID: \"a6752a30-3c25-4979-b28e-b45d83f10eba\") " pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:31.958134 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956966    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-modprobe-d\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956991    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e10f70fd-1430-4819-b75b-a236e679fb6f-konnectivity-ca\") pod \"konnectivity-agent-g8grv\" (UID: \"e10f70fd-1430-4819-b75b-a236e679fb6f\") " pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957018    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-log-socket\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957043    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-cni-bin\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957069    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-system-cni-dir\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957092    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-systemd\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957113    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f5f9f31-0c01-410a-ba00-e9e2a20272be-tmp\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957151    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957162    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05477ef7-e414-40b8-9d2d-b7da68fd264f-cni-binary-copy\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957176    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-etc-openvswitch\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957222    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-etc-openvswitch\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957229    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-var-lib-kubelet\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.956942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da52533e-1455-4ff1-8942-6cf7633a85a1-serviceca\") pod \"node-ca-47jpr\" (UID: \"da52533e-1455-4ff1-8942-6cf7633a85a1\") " pod="openshift-image-registry/node-ca-47jpr"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957237    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957261    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-etc-kubernetes\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957288    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbfe671c-f26a-484b-8e4a-d634b1731f98-tmp-dir\") pod \"node-resolver-qgvf6\" (UID: \"dbfe671c-f26a-484b-8e4a-d634b1731f98\") " pod="openshift-dns/node-resolver-qgvf6"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957336    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-etc-kubernetes\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957373    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-log-socket\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959040 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957406    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-cni-bin\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957446    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-var-lib-kubelet\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957462    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-system-cni-dir\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-systemd-units\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957497    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-node-log\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957556    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-run-ovn\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957601    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-systemd-units\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957636    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-run-ovn\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957655    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05477ef7-e414-40b8-9d2d-b7da68fd264f-system-cni-dir\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957682    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05477ef7-e414-40b8-9d2d-b7da68fd264f-cnibin\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957711    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-kubelet\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957773    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05477ef7-e414-40b8-9d2d-b7da68fd264f-system-cni-dir\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957800    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-var-lib-openvswitch\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957814    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05477ef7-e414-40b8-9d2d-b7da68fd264f-cnibin\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-run-ovn-kubernetes\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957879    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/05477ef7-e414-40b8-9d2d-b7da68fd264f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957907    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-kubelet\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.959769 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957907    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-var-lib-cni-multus\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957935    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m6xr\" (UniqueName: \"kubernetes.io/projected/a9fa416b-05ee-4410-a741-c433bde86ee0-kube-api-access-4m6xr\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957943    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-host-run-ovn-kubernetes\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fad7502f-aea7-4f92-9cd2-7bc9d4df2a62-host-slash\") pod \"iptables-alerter-kdhc6\" (UID: \"fad7502f-aea7-4f92-9cd2-7bc9d4df2a62\") " pod="openshift-network-operator/iptables-alerter-kdhc6"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957970    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7869bcf-f35e-43bf-baa4-04fe6606396a-cni-binary-copy\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957985    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-host-var-lib-cni-multus\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957985    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-run\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.957884    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-var-lib-openvswitch\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.958014    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-run-systemd\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.958039    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56gsj\" (UniqueName: \"kubernetes.io/projected/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-kube-api-access-56gsj\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.958068    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-run-systemd\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.958107    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/05477ef7-e414-40b8-9d2d-b7da68fd264f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.958221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-cni-dir\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.958306    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/05477ef7-e414-40b8-9d2d-b7da68fd264f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.958354    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-cni-dir\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.958485    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-ovnkube-script-lib\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.958682    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/05477ef7-e414-40b8-9d2d-b7da68fd264f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:31.960474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.958734    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-env-overrides\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.962734 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.959059    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7869bcf-f35e-43bf-baa4-04fe6606396a-multus-daemon-config\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.962734 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.960807    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-79.ec2.internal" event={"ID":"24342c83cc108d3ad0b5e1eda2e8279a","Type":"ContainerStarted","Data":"2fd0c04761f0896fd6901bb96beecc4d03cadd1525195f5e90b88627b194eb56"}
Sep 08 13:58:31.962734 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.961189    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-ovn-node-metrics-cert\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.971995 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.971960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crmxx\" (UniqueName: \"kubernetes.io/projected/aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8-kube-api-access-crmxx\") pod \"aws-ebs-csi-driver-node-79pxj\" (UID: \"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:31.972545 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.972157    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncvf9\" (UniqueName: \"kubernetes.io/projected/da52533e-1455-4ff1-8942-6cf7633a85a1-kube-api-access-ncvf9\") pod \"node-ca-47jpr\" (UID: \"da52533e-1455-4ff1-8942-6cf7633a85a1\") " pod="openshift-image-registry/node-ca-47jpr"
Sep 08 13:58:31.972545 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.972506    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56gsj\" (UniqueName: \"kubernetes.io/projected/8d73c1ac-fd9b-4432-a67f-1ce29ef0066d-kube-api-access-56gsj\") pod \"ovnkube-node-zsjl2\" (UID: \"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d\") " pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:31.973463 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.973442    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzgwq\" (UniqueName: \"kubernetes.io/projected/f7869bcf-f35e-43bf-baa4-04fe6606396a-kube-api-access-kzgwq\") pod \"multus-gvwgp\" (UID: \"f7869bcf-f35e-43bf-baa4-04fe6606396a\") " pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:31.974289 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:31.974268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5zjb\" (UniqueName: \"kubernetes.io/projected/05477ef7-e414-40b8-9d2d-b7da68fd264f-kube-api-access-l5zjb\") pod \"multus-additional-cni-plugins-nhsp9\" (UID: \"05477ef7-e414-40b8-9d2d-b7da68fd264f\") " pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:32.059187 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059150    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m6xr\" (UniqueName: \"kubernetes.io/projected/a9fa416b-05ee-4410-a741-c433bde86ee0-kube-api-access-4m6xr\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:32.059371 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059197    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fad7502f-aea7-4f92-9cd2-7bc9d4df2a62-host-slash\") pod \"iptables-alerter-kdhc6\" (UID: \"fad7502f-aea7-4f92-9cd2-7bc9d4df2a62\") " pod="openshift-network-operator/iptables-alerter-kdhc6"
Sep 08 13:58:32.059371 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059220    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-run\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.059371 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059268    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:32.059371 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059308    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-run\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.059371 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059309    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-sysconfig\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.059371 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059357    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-kubernetes\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.059371 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059365    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-sysconfig\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.059668 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059386    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-var-lib-kubelet\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.059668 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.059404    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:32.059668 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-var-lib-kubelet\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.059668 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-kubernetes\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.059668 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.059498    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs podName:a9fa416b-05ee-4410-a741-c433bde86ee0 nodeName:}" failed. No retries permitted until 2026-09-08 13:58:32.559473889 +0000 UTC m=+3.054444567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs") pod "network-metrics-daemon-zkd7c" (UID: "a9fa416b-05ee-4410-a741-c433bde86ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:32.059668 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059523    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fad7502f-aea7-4f92-9cd2-7bc9d4df2a62-host-slash\") pod \"iptables-alerter-kdhc6\" (UID: \"fad7502f-aea7-4f92-9cd2-7bc9d4df2a62\") " pod="openshift-network-operator/iptables-alerter-kdhc6"
Sep 08 13:58:32.059668 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059562    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srqg2\" (UniqueName: \"kubernetes.io/projected/6f5f9f31-0c01-410a-ba00-e9e2a20272be-kube-api-access-srqg2\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.059668 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55cm4\" (UniqueName: \"kubernetes.io/projected/dbfe671c-f26a-484b-8e4a-d634b1731f98-kube-api-access-55cm4\") pod \"node-resolver-qgvf6\" (UID: \"dbfe671c-f26a-484b-8e4a-d634b1731f98\") " pod="openshift-dns/node-resolver-qgvf6"
Sep 08 13:58:32.059668 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fad7502f-aea7-4f92-9cd2-7bc9d4df2a62-iptables-alerter-script\") pod \"iptables-alerter-kdhc6\" (UID: \"fad7502f-aea7-4f92-9cd2-7bc9d4df2a62\") " pod="openshift-network-operator/iptables-alerter-kdhc6"
Sep 08 13:58:32.059668 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059636    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-sys\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059698    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-sys\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059743    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-host\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-sysctl-d\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059775    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-host\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059793    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-sysctl-conf\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059808    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2b891b6-0ff7-4267-b030-ee95f25a0727-dbus\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkbd4\" (UniqueName: \"kubernetes.io/projected/fad7502f-aea7-4f92-9cd2-7bc9d4df2a62-kube-api-access-hkbd4\") pod \"iptables-alerter-kdhc6\" (UID: \"fad7502f-aea7-4f92-9cd2-7bc9d4df2a62\") " pod="openshift-network-operator/iptables-alerter-kdhc6"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059838    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2b891b6-0ff7-4267-b030-ee95f25a0727-kubelet-config\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059881    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-tuned\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059904    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e10f70fd-1430-4819-b75b-a236e679fb6f-agent-certs\") pod \"konnectivity-agent-g8grv\" (UID: \"e10f70fd-1430-4819-b75b-a236e679fb6f\") " pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059909    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c2b891b6-0ff7-4267-b030-ee95f25a0727-dbus\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-lib-modules\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059932    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c2b891b6-0ff7-4267-b030-ee95f25a0727-kubelet-config\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059948    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-sysctl-conf\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059957    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dbfe671c-f26a-484b-8e4a-d634b1731f98-hosts-file\") pod \"node-resolver-qgvf6\" (UID: \"dbfe671c-f26a-484b-8e4a-d634b1731f98\") " pod="openshift-dns/node-resolver-qgvf6"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.059917    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-sysctl-d\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060000    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbnm7\" (UniqueName: \"kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7\") pod \"network-check-target-5mbt5\" (UID: \"a6752a30-3c25-4979-b28e-b45d83f10eba\") " pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:32.060214 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060006    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dbfe671c-f26a-484b-8e4a-d634b1731f98-hosts-file\") pod \"node-resolver-qgvf6\" (UID: \"dbfe671c-f26a-484b-8e4a-d634b1731f98\") " pod="openshift-dns/node-resolver-qgvf6"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060020    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-modprobe-d\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e10f70fd-1430-4819-b75b-a236e679fb6f-konnectivity-ca\") pod \"konnectivity-agent-g8grv\" (UID: \"e10f70fd-1430-4819-b75b-a236e679fb6f\") " pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060080    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-systemd\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060102    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f5f9f31-0c01-410a-ba00-e9e2a20272be-tmp\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060124    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060157    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbfe671c-f26a-484b-8e4a-d634b1731f98-tmp-dir\") pod \"node-resolver-qgvf6\" (UID: \"dbfe671c-f26a-484b-8e4a-d634b1731f98\") " pod="openshift-dns/node-resolver-qgvf6"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060159    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-modprobe-d\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060080    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-lib-modules\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060224    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-systemd\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.060244    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fad7502f-aea7-4f92-9cd2-7bc9d4df2a62-iptables-alerter-script\") pod \"iptables-alerter-kdhc6\" (UID: \"fad7502f-aea7-4f92-9cd2-7bc9d4df2a62\") " pod="openshift-network-operator/iptables-alerter-kdhc6"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.060302    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret podName:c2b891b6-0ff7-4267-b030-ee95f25a0727 nodeName:}" failed. No retries permitted until 2026-09-08 13:58:32.5602843 +0000 UTC m=+3.055254966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret") pod "global-pull-secret-syncer-lfxpp" (UID: "c2b891b6-0ff7-4267-b030-ee95f25a0727") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060445    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbfe671c-f26a-484b-8e4a-d634b1731f98-tmp-dir\") pod \"node-resolver-qgvf6\" (UID: \"dbfe671c-f26a-484b-8e4a-d634b1731f98\") " pod="openshift-dns/node-resolver-qgvf6"
Sep 08 13:58:32.061266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.060577    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e10f70fd-1430-4819-b75b-a236e679fb6f-konnectivity-ca\") pod \"konnectivity-agent-g8grv\" (UID: \"e10f70fd-1430-4819-b75b-a236e679fb6f\") " pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:32.062534 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.062509    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f5f9f31-0c01-410a-ba00-e9e2a20272be-etc-tuned\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.062636 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.062578    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f5f9f31-0c01-410a-ba00-e9e2a20272be-tmp\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.062740 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.062722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e10f70fd-1430-4819-b75b-a236e679fb6f-agent-certs\") pod \"konnectivity-agent-g8grv\" (UID: \"e10f70fd-1430-4819-b75b-a236e679fb6f\") " pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:32.077038 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.076976    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:58:32.077038 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.077030    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:58:32.077208 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.077044    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wbnm7 for pod openshift-network-diagnostics/network-check-target-5mbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:32.077208 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.077110    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7 podName:a6752a30-3c25-4979-b28e-b45d83f10eba nodeName:}" failed. No retries permitted until 2026-09-08 13:58:32.577092794 +0000 UTC m=+3.072063457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wbnm7" (UniqueName: "kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7") pod "network-check-target-5mbt5" (UID: "a6752a30-3c25-4979-b28e-b45d83f10eba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:32.077622 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.077591    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srqg2\" (UniqueName: \"kubernetes.io/projected/6f5f9f31-0c01-410a-ba00-e9e2a20272be-kube-api-access-srqg2\") pod \"tuned-bqjtj\" (UID: \"6f5f9f31-0c01-410a-ba00-e9e2a20272be\") " pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.079233 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.079200    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkbd4\" (UniqueName: \"kubernetes.io/projected/fad7502f-aea7-4f92-9cd2-7bc9d4df2a62-kube-api-access-hkbd4\") pod \"iptables-alerter-kdhc6\" (UID: \"fad7502f-aea7-4f92-9cd2-7bc9d4df2a62\") " pod="openshift-network-operator/iptables-alerter-kdhc6"
Sep 08 13:58:32.079551 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.079532    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55cm4\" (UniqueName: \"kubernetes.io/projected/dbfe671c-f26a-484b-8e4a-d634b1731f98-kube-api-access-55cm4\") pod \"node-resolver-qgvf6\" (UID: \"dbfe671c-f26a-484b-8e4a-d634b1731f98\") " pod="openshift-dns/node-resolver-qgvf6"
Sep 08 13:58:32.081102 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.081078    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m6xr\" (UniqueName: \"kubernetes.io/projected/a9fa416b-05ee-4410-a741-c433bde86ee0-kube-api-access-4m6xr\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:32.147462 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.147434    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-47jpr"
Sep 08 13:58:32.154269 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.154246    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:32.163124 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.163102    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj"
Sep 08 13:58:32.167837 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.167809    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nhsp9"
Sep 08 13:58:32.174273 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.174247    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gvwgp"
Sep 08 13:58:32.182691 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.182670    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:32.189270 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.189249    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qgvf6"
Sep 08 13:58:32.196097 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.196079    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kdhc6"
Sep 08 13:58:32.201808 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.201789    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bqjtj"
Sep 08 13:58:32.477321 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:32.477285    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d73c1ac_fd9b_4432_a67f_1ce29ef0066d.slice/crio-a647658b0c28bbda3ffc4dd72bd664a798441d65b6e281f6b560885be24d040a WatchSource:0}: Error finding container a647658b0c28bbda3ffc4dd72bd664a798441d65b6e281f6b560885be24d040a: Status 404 returned error can't find the container with id a647658b0c28bbda3ffc4dd72bd664a798441d65b6e281f6b560885be24d040a
Sep 08 13:58:32.478037 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:32.478016    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed04cbd_4fdd_4cda_babd_13d0ecc7e6f8.slice/crio-19d447891db21e0f828a7c0c20ce472a053e8d92bcc357c1e01c2b16a9442014 WatchSource:0}: Error finding container 19d447891db21e0f828a7c0c20ce472a053e8d92bcc357c1e01c2b16a9442014: Status 404 returned error can't find the container with id 19d447891db21e0f828a7c0c20ce472a053e8d92bcc357c1e01c2b16a9442014
Sep 08 13:58:32.480986 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:32.479685    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5f9f31_0c01_410a_ba00_e9e2a20272be.slice/crio-3d699a930838685dd60ca1742a7aa85c9d87bacfc985cb6fab12e975dbff1c90 WatchSource:0}: Error finding container 3d699a930838685dd60ca1742a7aa85c9d87bacfc985cb6fab12e975dbff1c90: Status 404 returned error can't find the container with id 3d699a930838685dd60ca1742a7aa85c9d87bacfc985cb6fab12e975dbff1c90
Sep 08 13:58:32.482476 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:32.482454    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda52533e_1455_4ff1_8942_6cf7633a85a1.slice/crio-f4d67720617cbfcba6ca58e19c454de223042a321f6a496473b906e842fe4bc4 WatchSource:0}: Error finding container f4d67720617cbfcba6ca58e19c454de223042a321f6a496473b906e842fe4bc4: Status 404 returned error can't find the container with id f4d67720617cbfcba6ca58e19c454de223042a321f6a496473b906e842fe4bc4
Sep 08 13:58:32.483242 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:32.483168    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad7502f_aea7_4f92_9cd2_7bc9d4df2a62.slice/crio-6fbadb01e4da0d0c3f7cf55a49d6a9683f0350d857b376b2b3c21da2d6fd5c8d WatchSource:0}: Error finding container 6fbadb01e4da0d0c3f7cf55a49d6a9683f0350d857b376b2b3c21da2d6fd5c8d: Status 404 returned error can't find the container with id 6fbadb01e4da0d0c3f7cf55a49d6a9683f0350d857b376b2b3c21da2d6fd5c8d
Sep 08 13:58:32.484538 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:32.484518    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10f70fd_1430_4819_b75b_a236e679fb6f.slice/crio-6075e95daa98f0e66ba920d7c6ed4a2fabe1a4d5501646ed9f46f27a783c0972 WatchSource:0}: Error finding container 6075e95daa98f0e66ba920d7c6ed4a2fabe1a4d5501646ed9f46f27a783c0972: Status 404 returned error can't find the container with id 6075e95daa98f0e66ba920d7c6ed4a2fabe1a4d5501646ed9f46f27a783c0972
Sep 08 13:58:32.485104 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:32.485086    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7869bcf_f35e_43bf_baa4_04fe6606396a.slice/crio-5c50adb44e74cea74d2aaa0339940a120a7b42d965139be26098d5da50104064 WatchSource:0}: Error finding container 5c50adb44e74cea74d2aaa0339940a120a7b42d965139be26098d5da50104064: Status 404 returned error can't find the container with id 5c50adb44e74cea74d2aaa0339940a120a7b42d965139be26098d5da50104064
Sep 08 13:58:32.486454 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:32.486424    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05477ef7_e414_40b8_9d2d_b7da68fd264f.slice/crio-ce332e9d7fa02940b7e606f93524b9906e7932695cc389a7e69cfa206cd1607f WatchSource:0}: Error finding container ce332e9d7fa02940b7e606f93524b9906e7932695cc389a7e69cfa206cd1607f: Status 404 returned error can't find the container with id ce332e9d7fa02940b7e606f93524b9906e7932695cc389a7e69cfa206cd1607f
Sep 08 13:58:32.509074 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:32.508994    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbfe671c_f26a_484b_8e4a_d634b1731f98.slice/crio-5e45eafbde510cc40c30e2ab8346f67759d7755e248756d3c741614e6351b758 WatchSource:0}: Error finding container 5e45eafbde510cc40c30e2ab8346f67759d7755e248756d3c741614e6351b758: Status 404 returned error can't find the container with id 5e45eafbde510cc40c30e2ab8346f67759d7755e248756d3c741614e6351b758
Sep 08 13:58:32.529189 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:58:32.529168    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c409c5eae75092b9e98660579ed5eaf.slice/crio-e1ca6db4c888a6665747422f9e08168e06f673f652575d0c33b7b9a9edf19ebe WatchSource:0}: Error finding container e1ca6db4c888a6665747422f9e08168e06f673f652575d0c33b7b9a9edf19ebe: Status 404 returned error can't find the container with id e1ca6db4c888a6665747422f9e08168e06f673f652575d0c33b7b9a9edf19ebe
Sep 08 13:58:32.563474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.563451    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:32.563556 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.563501    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:32.563591 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.563583    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:32.563627 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.563584    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:32.563660 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.563626    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret podName:c2b891b6-0ff7-4267-b030-ee95f25a0727 nodeName:}" failed. No retries permitted until 2026-09-08 13:58:33.563613955 +0000 UTC m=+4.058584617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret") pod "global-pull-secret-syncer-lfxpp" (UID: "c2b891b6-0ff7-4267-b030-ee95f25a0727") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:32.563660 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.563639    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs podName:a9fa416b-05ee-4410-a741-c433bde86ee0 nodeName:}" failed. No retries permitted until 2026-09-08 13:58:33.563633152 +0000 UTC m=+4.058603814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs") pod "network-metrics-daemon-zkd7c" (UID: "a9fa416b-05ee-4410-a741-c433bde86ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:32.664756 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.664719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbnm7\" (UniqueName: \"kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7\") pod \"network-check-target-5mbt5\" (UID: \"a6752a30-3c25-4979-b28e-b45d83f10eba\") " pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:32.664914 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.664874    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:58:32.664914 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.664891    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:58:32.664914 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.664900    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wbnm7 for pod openshift-network-diagnostics/network-check-target-5mbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:32.665057 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:32.664947    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7 podName:a6752a30-3c25-4979-b28e-b45d83f10eba nodeName:}" failed. No retries permitted until 2026-09-08 13:58:33.664934592 +0000 UTC m=+4.159905254 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbnm7" (UniqueName: "kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7") pod "network-check-target-5mbt5" (UID: "a6752a30-3c25-4979-b28e-b45d83f10eba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:32.905985 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.905849    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 13:53:31 +0000 UTC" deadline="2028-05-07 07:10:18.563206113 +0000 UTC"
Sep 08 13:58:32.905985 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.905928    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14561h11m45.657282097s"
Sep 08 13:58:32.968895 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.968846    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal" event={"ID":"6c409c5eae75092b9e98660579ed5eaf","Type":"ContainerStarted","Data":"e1ca6db4c888a6665747422f9e08168e06f673f652575d0c33b7b9a9edf19ebe"}
Sep 08 13:58:32.970698 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.970666    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhsp9" event={"ID":"05477ef7-e414-40b8-9d2d-b7da68fd264f","Type":"ContainerStarted","Data":"ce332e9d7fa02940b7e606f93524b9906e7932695cc389a7e69cfa206cd1607f"}
Sep 08 13:58:32.972846 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.972822    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gvwgp" event={"ID":"f7869bcf-f35e-43bf-baa4-04fe6606396a","Type":"ContainerStarted","Data":"5c50adb44e74cea74d2aaa0339940a120a7b42d965139be26098d5da50104064"}
Sep 08 13:58:32.979725 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.979698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g8grv" event={"ID":"e10f70fd-1430-4819-b75b-a236e679fb6f","Type":"ContainerStarted","Data":"6075e95daa98f0e66ba920d7c6ed4a2fabe1a4d5501646ed9f46f27a783c0972"}
Sep 08 13:58:32.981596 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.981568    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kdhc6" event={"ID":"fad7502f-aea7-4f92-9cd2-7bc9d4df2a62","Type":"ContainerStarted","Data":"6fbadb01e4da0d0c3f7cf55a49d6a9683f0350d857b376b2b3c21da2d6fd5c8d"}
Sep 08 13:58:32.983312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.983285    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-47jpr" event={"ID":"da52533e-1455-4ff1-8942-6cf7633a85a1","Type":"ContainerStarted","Data":"f4d67720617cbfcba6ca58e19c454de223042a321f6a496473b906e842fe4bc4"}
Sep 08 13:58:32.984996 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.984972    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bqjtj" event={"ID":"6f5f9f31-0c01-410a-ba00-e9e2a20272be","Type":"ContainerStarted","Data":"3d699a930838685dd60ca1742a7aa85c9d87bacfc985cb6fab12e975dbff1c90"}
Sep 08 13:58:32.986748 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.986723    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2" event={"ID":"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d","Type":"ContainerStarted","Data":"a647658b0c28bbda3ffc4dd72bd664a798441d65b6e281f6b560885be24d040a"}
Sep 08 13:58:32.990450 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.990423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-79.ec2.internal" event={"ID":"24342c83cc108d3ad0b5e1eda2e8279a","Type":"ContainerStarted","Data":"4ddcbabe02ab6053312cfdf8ae6fc89e27d0cf6b6abf90490388f0ef7133b57b"}
Sep 08 13:58:32.994129 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.994107    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qgvf6" event={"ID":"dbfe671c-f26a-484b-8e4a-d634b1731f98","Type":"ContainerStarted","Data":"5e45eafbde510cc40c30e2ab8346f67759d7755e248756d3c741614e6351b758"}
Sep 08 13:58:32.995486 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:32.995454    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj" event={"ID":"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8","Type":"ContainerStarted","Data":"19d447891db21e0f828a7c0c20ce472a053e8d92bcc357c1e01c2b16a9442014"}
Sep 08 13:58:33.084988 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:33.084954    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:58:33.572021 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:33.571979    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:33.572206 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:33.572048    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:33.572206 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:33.572192    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:33.572314 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:33.572253    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs podName:a9fa416b-05ee-4410-a741-c433bde86ee0 nodeName:}" failed. No retries permitted until 2026-09-08 13:58:35.572234594 +0000 UTC m=+6.067205280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs") pod "network-metrics-daemon-zkd7c" (UID: "a9fa416b-05ee-4410-a741-c433bde86ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:33.572765 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:33.572646    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:33.572765 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:33.572698    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret podName:c2b891b6-0ff7-4267-b030-ee95f25a0727 nodeName:}" failed. No retries permitted until 2026-09-08 13:58:35.572683451 +0000 UTC m=+6.067654147 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret") pod "global-pull-secret-syncer-lfxpp" (UID: "c2b891b6-0ff7-4267-b030-ee95f25a0727") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:33.673346 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:33.673309    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbnm7\" (UniqueName: \"kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7\") pod \"network-check-target-5mbt5\" (UID: \"a6752a30-3c25-4979-b28e-b45d83f10eba\") " pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:33.673525 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:33.673484    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:58:33.673525 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:33.673506    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:58:33.673525 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:33.673520    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wbnm7 for pod openshift-network-diagnostics/network-check-target-5mbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:33.673683 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:33.673579    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7 podName:a6752a30-3c25-4979-b28e-b45d83f10eba nodeName:}" failed. No retries permitted until 2026-09-08 13:58:35.673561171 +0000 UTC m=+6.168531856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbnm7" (UniqueName: "kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7") pod "network-check-target-5mbt5" (UID: "a6752a30-3c25-4979-b28e-b45d83f10eba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:33.907016 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:33.906851    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 13:53:31 +0000 UTC" deadline="2028-02-07 11:35:07.332624126 +0000 UTC"
Sep 08 13:58:33.907016 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:33.906908    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12405h36m33.425719907s"
Sep 08 13:58:33.953536 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:33.953505    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:33.953729 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:33.953635    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:33.953797 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:33.953736    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:33.953850 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:33.953805    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:33.953919 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:33.953887    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:33.953990 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:33.953972    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:35.590098 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:35.590063    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:35.590545 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:35.590112    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:35.590545 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:35.590226    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:35.590545 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:35.590228    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:35.590545 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:35.590308    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs podName:a9fa416b-05ee-4410-a741-c433bde86ee0 nodeName:}" failed. No retries permitted until 2026-09-08 13:58:39.590267532 +0000 UTC m=+10.085238194 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs") pod "network-metrics-daemon-zkd7c" (UID: "a9fa416b-05ee-4410-a741-c433bde86ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:35.590545 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:35.590323    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret podName:c2b891b6-0ff7-4267-b030-ee95f25a0727 nodeName:}" failed. No retries permitted until 2026-09-08 13:58:39.590316504 +0000 UTC m=+10.085287166 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret") pod "global-pull-secret-syncer-lfxpp" (UID: "c2b891b6-0ff7-4267-b030-ee95f25a0727") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:35.691435 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:35.691394    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbnm7\" (UniqueName: \"kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7\") pod \"network-check-target-5mbt5\" (UID: \"a6752a30-3c25-4979-b28e-b45d83f10eba\") " pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:35.691627 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:35.691604    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:58:35.691627 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:35.691627    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:58:35.691749 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:35.691639    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wbnm7 for pod openshift-network-diagnostics/network-check-target-5mbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:35.691749 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:35.691696    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7 podName:a6752a30-3c25-4979-b28e-b45d83f10eba nodeName:}" failed. No retries permitted until 2026-09-08 13:58:39.691676454 +0000 UTC m=+10.186647139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbnm7" (UniqueName: "kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7") pod "network-check-target-5mbt5" (UID: "a6752a30-3c25-4979-b28e-b45d83f10eba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:35.956091 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:35.955219    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:35.956091 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:35.955360    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:35.956091 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:35.955748    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:35.956091 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:35.955843    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:35.956091 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:35.955924    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:35.956091 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:35.955996    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:37.952985 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:37.952949    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:37.953477 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:37.953088    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:37.953477 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:37.953461    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:37.953556 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:37.953534    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:37.953593 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:37.953573    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:37.953630 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:37.953620    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:39.624523 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:39.624479    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:39.624981 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:39.624543    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:39.624981 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:39.624675    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:39.624981 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:39.624741    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs podName:a9fa416b-05ee-4410-a741-c433bde86ee0 nodeName:}" failed. No retries permitted until 2026-09-08 13:58:47.624722188 +0000 UTC m=+18.119692854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs") pod "network-metrics-daemon-zkd7c" (UID: "a9fa416b-05ee-4410-a741-c433bde86ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:39.625124 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:39.625100    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:39.625196 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:39.625142    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret podName:c2b891b6-0ff7-4267-b030-ee95f25a0727 nodeName:}" failed. No retries permitted until 2026-09-08 13:58:47.625131578 +0000 UTC m=+18.120102240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret") pod "global-pull-secret-syncer-lfxpp" (UID: "c2b891b6-0ff7-4267-b030-ee95f25a0727") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:39.725948 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:39.725904    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbnm7\" (UniqueName: \"kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7\") pod \"network-check-target-5mbt5\" (UID: \"a6752a30-3c25-4979-b28e-b45d83f10eba\") " pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:39.726239 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:39.726223    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:58:39.726289 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:39.726246    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:58:39.726289 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:39.726260    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wbnm7 for pod openshift-network-diagnostics/network-check-target-5mbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:39.726366 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:39.726321    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7 podName:a6752a30-3c25-4979-b28e-b45d83f10eba nodeName:}" failed. No retries permitted until 2026-09-08 13:58:47.726302832 +0000 UTC m=+18.221273499 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbnm7" (UniqueName: "kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7") pod "network-check-target-5mbt5" (UID: "a6752a30-3c25-4979-b28e-b45d83f10eba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:39.953100 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:39.953020    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:39.953248 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:39.953148    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:39.954358 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:39.954218    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:39.954358 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:39.954316    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:39.954548 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:39.954355    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:39.954548 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:39.954483    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:41.953813 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:41.953136    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:41.953813 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:41.953268    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:41.953813 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:41.953136    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:41.953813 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:41.953394    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:41.953813 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:41.953137    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:41.953813 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:41.953480    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:43.953204 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:43.953170    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:43.953204 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:43.953195    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:43.953680 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:43.953180    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:43.953680 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:43.953308    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:43.953680 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:43.953402    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:43.953680 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:43.953500    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:45.952687 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:45.952648    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:45.953178 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:45.952648    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:45.953178 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:45.952780    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:45.953178 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:45.952797    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:45.953178 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:45.952922    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:45.953178 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:45.953028    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:47.686277 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:47.686243    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:47.686833 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:47.686322    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:47.686833 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:47.686413    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:47.686833 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:47.686414    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:47.686833 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:47.686463    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret podName:c2b891b6-0ff7-4267-b030-ee95f25a0727 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:03.68645066 +0000 UTC m=+34.181421322 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret") pod "global-pull-secret-syncer-lfxpp" (UID: "c2b891b6-0ff7-4267-b030-ee95f25a0727") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:58:47.686833 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:47.686478    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs podName:a9fa416b-05ee-4410-a741-c433bde86ee0 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:03.686471124 +0000 UTC m=+34.181441786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs") pod "network-metrics-daemon-zkd7c" (UID: "a9fa416b-05ee-4410-a741-c433bde86ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:58:47.786672 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:47.786638    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbnm7\" (UniqueName: \"kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7\") pod \"network-check-target-5mbt5\" (UID: \"a6752a30-3c25-4979-b28e-b45d83f10eba\") " pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:47.786818 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:47.786759    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:58:47.786818 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:47.786786    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:58:47.786818 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:47.786802    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wbnm7 for pod openshift-network-diagnostics/network-check-target-5mbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:47.786993 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:47.786878    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7 podName:a6752a30-3c25-4979-b28e-b45d83f10eba nodeName:}" failed. No retries permitted until 2026-09-08 13:59:03.786846239 +0000 UTC m=+34.281816901 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbnm7" (UniqueName: "kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7") pod "network-check-target-5mbt5" (UID: "a6752a30-3c25-4979-b28e-b45d83f10eba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:58:47.952684 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:47.952606    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:47.952684 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:47.952626    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:47.952912 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:47.952620    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:47.952912 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:47.952741    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:47.952912 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:47.952803    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:47.952912 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:47.952883    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:49.953954 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:49.953918    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:49.954420 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:49.953982    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:49.954420 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:49.954091    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:49.954420 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:49.954131    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:49.954420 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:49.954181    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:49.954420 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:49.954297    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:51.044938 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:51.044891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gvwgp" event={"ID":"f7869bcf-f35e-43bf-baa4-04fe6606396a","Type":"ContainerStarted","Data":"c7f1501959017f385ee885e6ed5210cd543674bb79f00b49c487b8cd2ffd4be5"}
Sep 08 13:58:51.048157 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:51.048111    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bqjtj" event={"ID":"6f5f9f31-0c01-410a-ba00-e9e2a20272be","Type":"ContainerStarted","Data":"02fda113dd0267d1d800a9ffc970086bf7605698f73f950977cafe533b8f862b"}
Sep 08 13:58:51.053986 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:51.053556    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2" event={"ID":"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d","Type":"ContainerStarted","Data":"b0f71391cbde746a7646b8dcd229baf50c14f4d08b81414167c9d0dc59478502"}
Sep 08 13:58:51.069453 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:51.069202    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gvwgp" podStartSLOduration=2.686229841 podStartE2EDuration="21.069182406s" totalImagesPullingTime="18.382952565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:51.069068393 +0000 UTC m=+21.564039078" watchObservedRunningTime="2026-09-08 13:58:51.069182406 +0000 UTC m=+21.564153091"
Sep 08 13:58:51.070048 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:51.070008    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-79.ec2.internal" podStartSLOduration=20.06999265 podStartE2EDuration="20.06999265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:58:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:33.012846743 +0000 UTC m=+3.507817428" watchObservedRunningTime="2026-09-08 13:58:51.06999265 +0000 UTC m=+21.564963335"
Sep 08 13:58:51.116895 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:51.116062    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bqjtj" podStartSLOduration=2.730268132 podStartE2EDuration="21.116039485s" totalImagesPullingTime="18.385771353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:51.115343551 +0000 UTC m=+21.610314237" watchObservedRunningTime="2026-09-08 13:58:51.116039485 +0000 UTC m=+21.611010170"
Sep 08 13:58:51.953406 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:51.953367    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:51.953406 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:51.953398    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:51.953624 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:51.953367    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:51.953624 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:51.953522    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:51.953717 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:51.953638    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:51.953717 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:51.953703    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:52.057078 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.057050    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kdhc6" event={"ID":"fad7502f-aea7-4f92-9cd2-7bc9d4df2a62","Type":"ContainerStarted","Data":"b29aa1eb37fe44c9b834093a66d143f2f4671c1f5ec34f123ccaf0538360657b"}
Sep 08 13:58:52.058553 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.058525    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-47jpr" event={"ID":"da52533e-1455-4ff1-8942-6cf7633a85a1","Type":"ContainerStarted","Data":"4c70ebeeec6aab44a065cdb7bc22cd93ac2a6586fd299326546e0f678ac79405"}
Sep 08 13:58:52.061377 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.061337    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2" event={"ID":"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d","Type":"ContainerStarted","Data":"627dab19ecf4f3c73716f367bbd8838f6519136cda8b467deccbdcdde6d51651"}
Sep 08 13:58:52.061495 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.061473    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2" event={"ID":"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d","Type":"ContainerStarted","Data":"d3c1889cb75a3585fb4bac1020c168a14c530358b93d56e5362d68d40a870770"}
Sep 08 13:58:52.061578 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.061569    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2" event={"ID":"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d","Type":"ContainerStarted","Data":"c24abd3f4e2f4530cfddde19273e9e6f46e08c4677836cc9885fc8cf8d1c0170"}
Sep 08 13:58:52.061636 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.061623    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2" event={"ID":"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d","Type":"ContainerStarted","Data":"6feb03ef5ec6f148230ebd3975c1e2e30dd7d10a442b80260ee85042fce370b4"}
Sep 08 13:58:52.061694 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.061671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2" event={"ID":"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d","Type":"ContainerStarted","Data":"238c5668ae6ca4f8b1f131b8b878a700dc439c861539b17d5f49d69130eded5d"}
Sep 08 13:58:52.062809 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.062785    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qgvf6" event={"ID":"dbfe671c-f26a-484b-8e4a-d634b1731f98","Type":"ContainerStarted","Data":"8a5d795fceccac64f7c18576383025164fa706919d4a8f10d9845a1eebea9e99"}
Sep 08 13:58:52.064272 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.064251    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj" event={"ID":"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8","Type":"ContainerStarted","Data":"b452b8101876be17ea90ac59ddf562ac56fef5cd4a7ccf758d81a68d253db193"}
Sep 08 13:58:52.065682 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.065661    2564 generic.go:356] "Generic (PLEG): container finished" podID="6c409c5eae75092b9e98660579ed5eaf" containerID="35893baba4ece346f12b3752706ea784b6e17a2f32387799e7f6e951c953be7e" exitCode=0
Sep 08 13:58:52.065770 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.065722    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal" event={"ID":"6c409c5eae75092b9e98660579ed5eaf","Type":"ContainerDied","Data":"35893baba4ece346f12b3752706ea784b6e17a2f32387799e7f6e951c953be7e"}
Sep 08 13:58:52.067276 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.067259    2564 generic.go:356] "Generic (PLEG): container finished" podID="05477ef7-e414-40b8-9d2d-b7da68fd264f" containerID="e942cee56200416a84fd8ebdb4ad26e5458d3575b91e9eba57c0133eb7d6c2fb" exitCode=0
Sep 08 13:58:52.067359 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.067319    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhsp9" event={"ID":"05477ef7-e414-40b8-9d2d-b7da68fd264f","Type":"ContainerDied","Data":"e942cee56200416a84fd8ebdb4ad26e5458d3575b91e9eba57c0133eb7d6c2fb"}
Sep 08 13:58:52.068853 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.068830    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g8grv" event={"ID":"e10f70fd-1430-4819-b75b-a236e679fb6f","Type":"ContainerStarted","Data":"8303d548a6e97ce0d9f12f406049f98bc96cd450655ce1dab0b1d6d87800734f"}
Sep 08 13:58:52.103788 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.103736    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-47jpr" podStartSLOduration=3.748651239 podStartE2EDuration="22.103717428s" totalImagesPullingTime="18.355066189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:52.086609976 +0000 UTC m=+22.581580661" watchObservedRunningTime="2026-09-08 13:58:52.103717428 +0000 UTC m=+22.598688112"
Sep 08 13:58:52.121070 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.120790    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qgvf6" podStartSLOduration=3.795853061 podStartE2EDuration="22.120777425s" totalImagesPullingTime="18.324924364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:52.120388133 +0000 UTC m=+22.615358817" watchObservedRunningTime="2026-09-08 13:58:52.120777425 +0000 UTC m=+22.615748108"
Sep 08 13:58:52.121298 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.121258    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g8grv" podStartSLOduration=3.789226517 podStartE2EDuration="22.121246495s" totalImagesPullingTime="18.332019978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:52.10414654 +0000 UTC m=+22.599117226" watchObservedRunningTime="2026-09-08 13:58:52.121246495 +0000 UTC m=+22.616217179"
Sep 08 13:58:52.725586 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.725560    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 13:58:52.916642 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.916475    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T13:58:52.725580875Z","UUID":"204d7733-77da-4ef2-bdb7-e09b57cba494","Handler":null,"Name":"","Endpoint":""}
Sep 08 13:58:52.918145 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.918128    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 13:58:52.918270 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:52.918161    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 13:58:53.073523 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:53.073479    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj" event={"ID":"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8","Type":"ContainerStarted","Data":"d255eac4537f4e1aa7be9abe96941d742cd3a07d84803f2e837b418a888d0383"}
Sep 08 13:58:53.076047 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:53.076004    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal" event={"ID":"6c409c5eae75092b9e98660579ed5eaf","Type":"ContainerStarted","Data":"5a1d6501cfcbcf834439de41032154c0dcf5e83a0a00b0ce2f3a9fd7b569cc91"}
Sep 08 13:58:53.110584 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:53.110534    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kdhc6" podStartSLOduration=4.7547670140000005 podStartE2EDuration="23.110513334s" totalImagesPullingTime="18.35574632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:53.11026888 +0000 UTC m=+23.605239563" watchObservedRunningTime="2026-09-08 13:58:53.110513334 +0000 UTC m=+23.605484018"
Sep 08 13:58:53.111077 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:53.111040    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-79.ec2.internal" podStartSLOduration=22.111028068 podStartE2EDuration="22.111028068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:58:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:53.093788064 +0000 UTC m=+23.588758751" watchObservedRunningTime="2026-09-08 13:58:53.111028068 +0000 UTC m=+23.605998755"
Sep 08 13:58:53.953510 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:53.953424    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:53.953510 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:53.953447    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:53.953705 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:53.953640    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:53.953705 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:53.953663    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:53.953827 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:53.953706    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:53.953827 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:53.953802    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:54.084171 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:54.084136    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2" event={"ID":"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d","Type":"ContainerStarted","Data":"308e9fa7a48cf99341ef89058dc8cede7d6bdfebad965883d2504294b3f65dca"}
Sep 08 13:58:54.923141 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:54.923103    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:54.923820 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:54.923796    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:55.088854 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:55.088799    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj" event={"ID":"aed04cbd-4fdd-4cda-babd-13d0ecc7e6f8","Type":"ContainerStarted","Data":"c3ae143074065b53d239883db5d6a1f4f2d47630628b13a9846be90b711950ad"}
Sep 08 13:58:55.953345 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:55.953316    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:55.953345 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:55.953341    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:55.953597 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:55.953315    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:55.953597 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:55.953443    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:55.953597 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:55.953513    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:55.953597 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:55.953580    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:57.097924 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.097638    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2" event={"ID":"8d73c1ac-fd9b-4432-a67f-1ce29ef0066d","Type":"ContainerStarted","Data":"e5171f2c98c8444774244ec682f176108f2699783adbf6971199063a1c512d3a"}
Sep 08 13:58:57.098551 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.097983    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:57.098551 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.098020    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:57.100266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.100219    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhsp9" event={"ID":"05477ef7-e414-40b8-9d2d-b7da68fd264f","Type":"ContainerStarted","Data":"9b998a6bb705184d277e1603ffdaf13bd72016080fb72c4c826db08a33923ca6"}
Sep 08 13:58:57.114361 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.114338    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:57.114880 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.114840    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:57.136165 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.136127    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-79pxj" podStartSLOduration=6.237208169 podStartE2EDuration="27.136111764s" totalImagesPullingTime="20.898903595s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:55.110834947 +0000 UTC m=+25.605805630" watchObservedRunningTime="2026-09-08 13:58:57.136111764 +0000 UTC m=+27.631082451"
Sep 08 13:58:57.173460 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.173387    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2" podStartSLOduration=8.762487027 podStartE2EDuration="27.173373893s" totalImagesPullingTime="18.410886866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:57.136956101 +0000 UTC m=+27.631926788" watchObservedRunningTime="2026-09-08 13:58:57.173373893 +0000 UTC m=+27.668344577"
Sep 08 13:58:57.768400 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.768368    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:58:57.952659 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.952625    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:57.952659 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.952655    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:57.952952 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:57.952636    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:57.952952 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:57.952758    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:57.952952 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:57.952857    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:57.952952 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:57.952949    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:58.104325 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:58.104229    2564 generic.go:356] "Generic (PLEG): container finished" podID="05477ef7-e414-40b8-9d2d-b7da68fd264f" containerID="9b998a6bb705184d277e1603ffdaf13bd72016080fb72c4c826db08a33923ca6" exitCode=0
Sep 08 13:58:58.104325 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:58.104303    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhsp9" event={"ID":"05477ef7-e414-40b8-9d2d-b7da68fd264f","Type":"ContainerDied","Data":"9b998a6bb705184d277e1603ffdaf13bd72016080fb72c4c826db08a33923ca6"}
Sep 08 13:58:58.702983 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:58.702654    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5mbt5"]
Sep 08 13:58:58.703151 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:58.703019    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:58:58.703151 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:58.703132    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:58:58.704521 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:58.704493    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lfxpp"]
Sep 08 13:58:58.704651 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:58.704599    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:58.704713 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:58.704692    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:58:58.708908 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:58.708882    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zkd7c"]
Sep 08 13:58:58.709021 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:58.708975    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:58:58.709081 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:58.709062    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:58:59.040086 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:59.039997    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:59.040252 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:59.040125    2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 08 13:58:59.040555 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:59.040535    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g8grv"
Sep 08 13:58:59.108480 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:59.108448    2564 generic.go:356] "Generic (PLEG): container finished" podID="05477ef7-e414-40b8-9d2d-b7da68fd264f" containerID="46b1ff20a13ea499a0b98e69f7b5ad7019551e26401f7d97c08b63288b27652b" exitCode=0
Sep 08 13:58:59.108982 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:59.108536    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhsp9" event={"ID":"05477ef7-e414-40b8-9d2d-b7da68fd264f","Type":"ContainerDied","Data":"46b1ff20a13ea499a0b98e69f7b5ad7019551e26401f7d97c08b63288b27652b"}
Sep 08 13:58:59.955312 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:58:59.955288    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:58:59.955426 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:58:59.955384    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:59:00.113608 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:00.113575    2564 generic.go:356] "Generic (PLEG): container finished" podID="05477ef7-e414-40b8-9d2d-b7da68fd264f" containerID="dc76a7558e0aa950734d5906bb4072ef1b5bc71987d2880e13d0d38738154fb5" exitCode=0
Sep 08 13:59:00.114573 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:00.113637    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhsp9" event={"ID":"05477ef7-e414-40b8-9d2d-b7da68fd264f","Type":"ContainerDied","Data":"dc76a7558e0aa950734d5906bb4072ef1b5bc71987d2880e13d0d38738154fb5"}
Sep 08 13:59:00.953223 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:00.953192    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:59:00.953405 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:00.953236    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:59:00.953405 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:00.953330    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:59:00.953515 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:00.953470    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:59:01.953434 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:01.953404    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:59:01.953921 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:01.953537    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lfxpp" podUID="c2b891b6-0ff7-4267-b030-ee95f25a0727"
Sep 08 13:59:02.952390 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:02.952357    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:59:02.952659 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:02.952488    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5mbt5" podUID="a6752a30-3c25-4979-b28e-b45d83f10eba"
Sep 08 13:59:02.952659 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:02.952554    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:59:02.952781 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:02.952724    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zkd7c" podUID="a9fa416b-05ee-4410-a741-c433bde86ee0"
Sep 08 13:59:03.701473 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.701386    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:59:03.701473 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.701429    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:59:03.701995 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:03.701563    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:59:03.701995 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:03.701593    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:59:03.701995 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:03.701644    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret podName:c2b891b6-0ff7-4267-b030-ee95f25a0727 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:35.701622446 +0000 UTC m=+66.196593125 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret") pod "global-pull-secret-syncer-lfxpp" (UID: "c2b891b6-0ff7-4267-b030-ee95f25a0727") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:59:03.701995 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:03.701664    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs podName:a9fa416b-05ee-4410-a741-c433bde86ee0 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:35.701654814 +0000 UTC m=+66.196625477 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs") pod "network-metrics-daemon-zkd7c" (UID: "a9fa416b-05ee-4410-a741-c433bde86ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:59:03.748685 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.748653    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-79.ec2.internal" event="NodeReady"
Sep 08 13:59:03.748907 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.748818    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 13:59:03.802071 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.802037    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbnm7\" (UniqueName: \"kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7\") pod \"network-check-target-5mbt5\" (UID: \"a6752a30-3c25-4979-b28e-b45d83f10eba\") " pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:59:03.802232 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:03.802161    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:59:03.802232 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:03.802179    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:59:03.802232 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:03.802192    2564 projected.go:196] Error preparing data for projected volume kube-api-access-wbnm7 for pod openshift-network-diagnostics/network-check-target-5mbt5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:59:03.802353 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:03.802243    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7 podName:a6752a30-3c25-4979-b28e-b45d83f10eba nodeName:}" failed. No retries permitted until 2026-09-08 13:59:35.802228643 +0000 UTC m=+66.297199309 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wbnm7" (UniqueName: "kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7") pod "network-check-target-5mbt5" (UID: "a6752a30-3c25-4979-b28e-b45d83f10eba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:59:03.825795 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.825754    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fwk9m"]
Sep 08 13:59:03.903257 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.903219    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qsrr6"]
Sep 08 13:59:03.903257 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.903249    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:03.907198 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.907167    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b4mdc\""
Sep 08 13:59:03.907328 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.907209    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 13:59:03.907505 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.907473    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 13:59:03.916159 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.916141    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fwk9m"]
Sep 08 13:59:03.916286 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.916170    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qsrr6"]
Sep 08 13:59:03.916286 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.916266    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qsrr6"
Sep 08 13:59:03.919425 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.919392    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7mkjh\""
Sep 08 13:59:03.919525 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.919438    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 13:59:03.919776 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.919758    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 13:59:03.920235 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.920217    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 13:59:03.955603 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.955547    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:59:03.960796 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:03.960616    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 13:59:04.004417 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.004386    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.004553 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.004429    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-tmp-dir\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.004553 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.004465    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4066c24-5beb-4011-a94b-cf594601f554-cert\") pod \"ingress-canary-qsrr6\" (UID: \"d4066c24-5beb-4011-a94b-cf594601f554\") " pod="openshift-ingress-canary/ingress-canary-qsrr6"
Sep 08 13:59:04.004553 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.004489    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-config-volume\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.004553 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.004524    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr8mb\" (UniqueName: \"kubernetes.io/projected/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-kube-api-access-tr8mb\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.004553 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.004549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt99f\" (UniqueName: \"kubernetes.io/projected/d4066c24-5beb-4011-a94b-cf594601f554-kube-api-access-dt99f\") pod \"ingress-canary-qsrr6\" (UID: \"d4066c24-5beb-4011-a94b-cf594601f554\") " pod="openshift-ingress-canary/ingress-canary-qsrr6"
Sep 08 13:59:04.104918 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.104882    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4066c24-5beb-4011-a94b-cf594601f554-cert\") pod \"ingress-canary-qsrr6\" (UID: \"d4066c24-5beb-4011-a94b-cf594601f554\") " pod="openshift-ingress-canary/ingress-canary-qsrr6"
Sep 08 13:59:04.105070 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.104927    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-config-volume\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.105070 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.104975    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr8mb\" (UniqueName: \"kubernetes.io/projected/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-kube-api-access-tr8mb\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.105070 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.104992    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt99f\" (UniqueName: \"kubernetes.io/projected/d4066c24-5beb-4011-a94b-cf594601f554-kube-api-access-dt99f\") pod \"ingress-canary-qsrr6\" (UID: \"d4066c24-5beb-4011-a94b-cf594601f554\") " pod="openshift-ingress-canary/ingress-canary-qsrr6"
Sep 08 13:59:04.105070 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.105031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.105268 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:04.105112    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:59:04.105268 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:04.105159    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls podName:a313f4b1-b29f-4d5f-a580-5ed5bd99696d nodeName:}" failed. No retries permitted until 2026-09-08 13:59:04.605144906 +0000 UTC m=+35.100115567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls") pod "dns-default-fwk9m" (UID: "a313f4b1-b29f-4d5f-a580-5ed5bd99696d") : secret "dns-default-metrics-tls" not found
Sep 08 13:59:04.105268 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.105223    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-tmp-dir\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.105521 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.105499    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-tmp-dir\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.105581 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.105527    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-config-volume\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.109386 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.109271    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4066c24-5beb-4011-a94b-cf594601f554-cert\") pod \"ingress-canary-qsrr6\" (UID: \"d4066c24-5beb-4011-a94b-cf594601f554\") " pod="openshift-ingress-canary/ingress-canary-qsrr6"
Sep 08 13:59:04.119706 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.119684    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr8mb\" (UniqueName: \"kubernetes.io/projected/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-kube-api-access-tr8mb\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.120839 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.120820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt99f\" (UniqueName: \"kubernetes.io/projected/d4066c24-5beb-4011-a94b-cf594601f554-kube-api-access-dt99f\") pod \"ingress-canary-qsrr6\" (UID: \"d4066c24-5beb-4011-a94b-cf594601f554\") " pod="openshift-ingress-canary/ingress-canary-qsrr6"
Sep 08 13:59:04.224344 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.224258    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qsrr6"
Sep 08 13:59:04.609296 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.609262    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:04.609440 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:04.609408    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:59:04.609479 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:04.609471    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls podName:a313f4b1-b29f-4d5f-a580-5ed5bd99696d nodeName:}" failed. No retries permitted until 2026-09-08 13:59:05.609456116 +0000 UTC m=+36.104426778 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls") pod "dns-default-fwk9m" (UID: "a313f4b1-b29f-4d5f-a580-5ed5bd99696d") : secret "dns-default-metrics-tls" not found
Sep 08 13:59:04.637633 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.637601    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qsrr6"]
Sep 08 13:59:04.642613 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:59:04.642585    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4066c24_5beb_4011_a94b_cf594601f554.slice/crio-1c07d664bbdc30792e83ac5fc34c2255bd280cbb0c1cd8530aacce44b9e1e5b0 WatchSource:0}: Error finding container 1c07d664bbdc30792e83ac5fc34c2255bd280cbb0c1cd8530aacce44b9e1e5b0: Status 404 returned error can't find the container with id 1c07d664bbdc30792e83ac5fc34c2255bd280cbb0c1cd8530aacce44b9e1e5b0
Sep 08 13:59:04.953384 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.953286    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:59:04.954084 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.953468    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:59:04.959272 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.959245    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 13:59:04.959272 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.959262    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 13:59:04.959422 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.959267    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 13:59:04.959500 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.959485    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r4t74\""
Sep 08 13:59:04.959552 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:04.959538    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9cmv8\""
Sep 08 13:59:05.127702 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:05.127664    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qsrr6" event={"ID":"d4066c24-5beb-4011-a94b-cf594601f554","Type":"ContainerStarted","Data":"1c07d664bbdc30792e83ac5fc34c2255bd280cbb0c1cd8530aacce44b9e1e5b0"}
Sep 08 13:59:05.134068 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:05.134038    2564 generic.go:356] "Generic (PLEG): container finished" podID="05477ef7-e414-40b8-9d2d-b7da68fd264f" containerID="e8f4c351eaa4cb625f876d7663ba6011801dd31b524b3aa3b0c5b32a3269a74b" exitCode=0
Sep 08 13:59:05.134216 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:05.134091    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhsp9" event={"ID":"05477ef7-e414-40b8-9d2d-b7da68fd264f","Type":"ContainerDied","Data":"e8f4c351eaa4cb625f876d7663ba6011801dd31b524b3aa3b0c5b32a3269a74b"}
Sep 08 13:59:05.616033 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:05.615949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:05.616196 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:05.616097    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:59:05.616196 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:05.616160    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls podName:a313f4b1-b29f-4d5f-a580-5ed5bd99696d nodeName:}" failed. No retries permitted until 2026-09-08 13:59:07.616143149 +0000 UTC m=+38.111113811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls") pod "dns-default-fwk9m" (UID: "a313f4b1-b29f-4d5f-a580-5ed5bd99696d") : secret "dns-default-metrics-tls" not found
Sep 08 13:59:06.140771 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:06.140736    2564 generic.go:356] "Generic (PLEG): container finished" podID="05477ef7-e414-40b8-9d2d-b7da68fd264f" containerID="3f2f70cb10b2a814a8e0481c361be7cb0209d7506b6db683a6894247918f67e1" exitCode=0
Sep 08 13:59:06.141359 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:06.140841    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhsp9" event={"ID":"05477ef7-e414-40b8-9d2d-b7da68fd264f","Type":"ContainerDied","Data":"3f2f70cb10b2a814a8e0481c361be7cb0209d7506b6db683a6894247918f67e1"}
Sep 08 13:59:07.146985 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:07.146770    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nhsp9" event={"ID":"05477ef7-e414-40b8-9d2d-b7da68fd264f","Type":"ContainerStarted","Data":"3d5b883d8a4a7b2dbbefeef5625a1cb9ace7a2614df431399b58098f71e0b6c0"}
Sep 08 13:59:07.184413 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:07.184360    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nhsp9" podStartSLOduration=8.553880795 podStartE2EDuration="37.184340043s" totalImagesPullingTime="28.630459248s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:59:07.181666083 +0000 UTC m=+37.676636768" watchObservedRunningTime="2026-09-08 13:59:07.184340043 +0000 UTC m=+37.679310728"
Sep 08 13:59:07.629103 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:07.629061    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:07.629258 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:07.629214    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:59:07.629300 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:07.629277    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls podName:a313f4b1-b29f-4d5f-a580-5ed5bd99696d nodeName:}" failed. No retries permitted until 2026-09-08 13:59:11.629260915 +0000 UTC m=+42.124231576 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls") pod "dns-default-fwk9m" (UID: "a313f4b1-b29f-4d5f-a580-5ed5bd99696d") : secret "dns-default-metrics-tls" not found
Sep 08 13:59:08.150777 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:08.150734    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qsrr6" event={"ID":"d4066c24-5beb-4011-a94b-cf594601f554","Type":"ContainerStarted","Data":"8b16253966690a5106644427d3a185e02f18354dcba1e28ad77b8e34121e5840"}
Sep 08 13:59:08.174355 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:08.174267    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qsrr6" podStartSLOduration=2.739595041 podStartE2EDuration="5.174250701s" totalImagesPullingTime="2.43465566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:59:08.174146819 +0000 UTC m=+38.669117503" watchObservedRunningTime="2026-09-08 13:59:08.174250701 +0000 UTC m=+38.669221432"
Sep 08 13:59:11.658208 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:11.658166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:11.658585 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:11.658344    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:59:11.658585 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:11.658412    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls podName:a313f4b1-b29f-4d5f-a580-5ed5bd99696d nodeName:}" failed. No retries permitted until 2026-09-08 13:59:19.658396352 +0000 UTC m=+50.153367017 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls") pod "dns-default-fwk9m" (UID: "a313f4b1-b29f-4d5f-a580-5ed5bd99696d") : secret "dns-default-metrics-tls" not found
Sep 08 13:59:19.718671 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:19.718629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:19.719250 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:19.718809    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:59:19.719250 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:19.718961    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls podName:a313f4b1-b29f-4d5f-a580-5ed5bd99696d nodeName:}" failed. No retries permitted until 2026-09-08 13:59:35.71893532 +0000 UTC m=+66.213905986 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls") pod "dns-default-fwk9m" (UID: "a313f4b1-b29f-4d5f-a580-5ed5bd99696d") : secret "dns-default-metrics-tls" not found
Sep 08 13:59:29.119025 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:29.118994    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zsjl2"
Sep 08 13:59:34.210900 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.210847    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"]
Sep 08 13:59:34.242758 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.242731    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"]
Sep 08 13:59:34.242916 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.242833    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:34.245843 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.245825    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 08 13:59:34.246062 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.246050    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:59:34.247066 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.247050    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-s4s4v\""
Sep 08 13:59:34.249485 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.249465    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 08 13:59:34.320496 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.320466    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-blvrw\" (UID: \"c29aa849-42c2-4f57-9668-4f7435dddec6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:34.320658 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.320555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztbpb\" (UniqueName: \"kubernetes.io/projected/c29aa849-42c2-4f57-9668-4f7435dddec6-kube-api-access-ztbpb\") pod \"cluster-samples-operator-7cdcfcf76c-blvrw\" (UID: \"c29aa849-42c2-4f57-9668-4f7435dddec6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:34.338382 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.338352    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-7s6g6"]
Sep 08 13:59:34.358966 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.358945    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-db7b6f4f5-xv6ht"]
Sep 08 13:59:34.359103 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.359061    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.361730 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.361699    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8tn6h\""
Sep 08 13:59:34.361828 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.361787    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 08 13:59:34.361828 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.361803    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 08 13:59:34.362234 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.362217    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 08 13:59:34.362540 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.362523    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:59:34.369120 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.369103    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 08 13:59:34.370955 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.370938    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-7s6g6"]
Sep 08 13:59:34.371048 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.371035    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.373922 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.373907    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 08 13:59:34.374021 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.374005    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 08 13:59:34.374097 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.374037    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j2pct\""
Sep 08 13:59:34.374097 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.374048    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 08 13:59:34.380530 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.380508    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db7b6f4f5-xv6ht"]
Sep 08 13:59:34.393258 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.393234    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 08 13:59:34.421612 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.421583    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztbpb\" (UniqueName: \"kubernetes.io/projected/c29aa849-42c2-4f57-9668-4f7435dddec6-kube-api-access-ztbpb\") pod \"cluster-samples-operator-7cdcfcf76c-blvrw\" (UID: \"c29aa849-42c2-4f57-9668-4f7435dddec6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:34.421612 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.421612    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.421758 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.421631    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-ca-trust-extracted\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.421758 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.421647    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7xzq\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-kube-api-access-n7xzq\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.421758 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.421664    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c2f5dad-d885-4865-84ba-9a31338284b5-trusted-ca\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.421758 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.421691    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-bound-sa-token\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.421758 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.421737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98nw5\" (UniqueName: \"kubernetes.io/projected/7c2f5dad-d885-4865-84ba-9a31338284b5-kube-api-access-98nw5\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.422022 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.421933    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7c2f5dad-d885-4865-84ba-9a31338284b5-config\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.422022 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.421964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-blvrw\" (UID: \"c29aa849-42c2-4f57-9668-4f7435dddec6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:34.422022 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.421989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-image-registry-private-configuration\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.422135 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.422035    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-certificates\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.422135 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.422060    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-installation-pull-secrets\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.422135 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:34.422077    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 13:59:34.422135 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.422129    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-trusted-ca\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.422259 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:34.422153    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls podName:c29aa849-42c2-4f57-9668-4f7435dddec6 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:34.922134102 +0000 UTC m=+65.417104764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-blvrw" (UID: "c29aa849-42c2-4f57-9668-4f7435dddec6") : secret "samples-operator-tls" not found
Sep 08 13:59:34.422259 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.422240    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c2f5dad-d885-4865-84ba-9a31338284b5-serving-cert\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.449228 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.449205    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztbpb\" (UniqueName: \"kubernetes.io/projected/c29aa849-42c2-4f57-9668-4f7435dddec6-kube-api-access-ztbpb\") pod \"cluster-samples-operator-7cdcfcf76c-blvrw\" (UID: \"c29aa849-42c2-4f57-9668-4f7435dddec6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:34.523371 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523337    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-bound-sa-token\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.523371 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523375    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98nw5\" (UniqueName: \"kubernetes.io/projected/7c2f5dad-d885-4865-84ba-9a31338284b5-kube-api-access-98nw5\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.523569 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523407    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7c2f5dad-d885-4865-84ba-9a31338284b5-config\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.523569 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523448    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-image-registry-private-configuration\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.523569 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523478    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-certificates\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.523569 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523500    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-installation-pull-secrets\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.523569 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523559    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-trusted-ca\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.523798 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c2f5dad-d885-4865-84ba-9a31338284b5-serving-cert\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.523798 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.523798 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523642    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-ca-trust-extracted\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.523798 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523663    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7xzq\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-kube-api-access-n7xzq\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.523798 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.523689    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c2f5dad-d885-4865-84ba-9a31338284b5-trusted-ca\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.523798 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:34.523761    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 13:59:34.523798 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:34.523779    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db7b6f4f5-xv6ht: secret "image-registry-tls" not found
Sep 08 13:59:34.524141 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:34.523845    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls podName:ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab nodeName:}" failed. No retries permitted until 2026-09-08 13:59:35.023824815 +0000 UTC m=+65.518795477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls") pod "image-registry-db7b6f4f5-xv6ht" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab") : secret "image-registry-tls" not found
Sep 08 13:59:34.524249 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.524224    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-certificates\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.524336 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.524315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7c2f5dad-d885-4865-84ba-9a31338284b5-config\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.524902 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.524875    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7c2f5dad-d885-4865-84ba-9a31338284b5-trusted-ca\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.525066 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.524976    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-trusted-ca\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.525066 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.525008    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-ca-trust-extracted\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.526318 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.526301    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-installation-pull-secrets\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.526404 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.526321    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c2f5dad-d885-4865-84ba-9a31338284b5-serving-cert\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.526442 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.526427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-image-registry-private-configuration\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.539553 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.539526    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98nw5\" (UniqueName: \"kubernetes.io/projected/7c2f5dad-d885-4865-84ba-9a31338284b5-kube-api-access-98nw5\") pod \"console-operator-85b454874b-7s6g6\" (UID: \"7c2f5dad-d885-4865-84ba-9a31338284b5\") " pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.539765 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.539750    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-bound-sa-token\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.542310 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.542284    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7xzq\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-kube-api-access-n7xzq\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:34.680323 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.680290    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:34.926137 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.926051    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-blvrw\" (UID: \"c29aa849-42c2-4f57-9668-4f7435dddec6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:34.926282 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:34.926194    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 13:59:34.926282 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:34.926257    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls podName:c29aa849-42c2-4f57-9668-4f7435dddec6 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:35.926241279 +0000 UTC m=+66.421211945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-blvrw" (UID: "c29aa849-42c2-4f57-9668-4f7435dddec6") : secret "samples-operator-tls" not found
Sep 08 13:59:34.991596 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:34.991564    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-7s6g6"]
Sep 08 13:59:34.994793 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:59:34.994762    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c2f5dad_d885_4865_84ba_9a31338284b5.slice/crio-a2d4cc06ba3eb0df3bbc3c034783d64415cb4efb9454b20602d90454ab989adf WatchSource:0}: Error finding container a2d4cc06ba3eb0df3bbc3c034783d64415cb4efb9454b20602d90454ab989adf: Status 404 returned error can't find the container with id a2d4cc06ba3eb0df3bbc3c034783d64415cb4efb9454b20602d90454ab989adf
Sep 08 13:59:35.026483 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.026463    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:35.026628 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:35.026610    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 13:59:35.026668 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:35.026630    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db7b6f4f5-xv6ht: secret "image-registry-tls" not found
Sep 08 13:59:35.026700 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:35.026681    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls podName:ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab nodeName:}" failed. No retries permitted until 2026-09-08 13:59:36.026666297 +0000 UTC m=+66.521636977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls") pod "image-registry-db7b6f4f5-xv6ht" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab") : secret "image-registry-tls" not found
Sep 08 13:59:35.215685 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.215654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-7s6g6" event={"ID":"7c2f5dad-d885-4865-84ba-9a31338284b5","Type":"ContainerStarted","Data":"a2d4cc06ba3eb0df3bbc3c034783d64415cb4efb9454b20602d90454ab989adf"}
Sep 08 13:59:35.732810 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.732768    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 13:59:35.733008 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.732837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 13:59:35.733008 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.732889    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:59:35.733008 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:35.732981    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:59:35.733111 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:35.733058    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls podName:a313f4b1-b29f-4d5f-a580-5ed5bd99696d nodeName:}" failed. No retries permitted until 2026-09-08 14:00:07.733040824 +0000 UTC m=+98.228011487 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls") pod "dns-default-fwk9m" (UID: "a313f4b1-b29f-4d5f-a580-5ed5bd99696d") : secret "dns-default-metrics-tls" not found
Sep 08 13:59:35.735451 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.735435    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 13:59:35.735633 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.735618    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 13:59:35.743031 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:35.743016    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 08 13:59:35.743087 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:35.743061    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs podName:a9fa416b-05ee-4410-a741-c433bde86ee0 nodeName:}" failed. No retries permitted until 2026-09-08 14:00:39.743048443 +0000 UTC m=+130.238019105 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs") pod "network-metrics-daemon-zkd7c" (UID: "a9fa416b-05ee-4410-a741-c433bde86ee0") : secret "metrics-daemon-secret" not found
Sep 08 13:59:35.745417 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.745396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c2b891b6-0ff7-4267-b030-ee95f25a0727-original-pull-secret\") pod \"global-pull-secret-syncer-lfxpp\" (UID: \"c2b891b6-0ff7-4267-b030-ee95f25a0727\") " pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:59:35.763129 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.763106    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lfxpp"
Sep 08 13:59:35.838766 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.836070    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbnm7\" (UniqueName: \"kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7\") pod \"network-check-target-5mbt5\" (UID: \"a6752a30-3c25-4979-b28e-b45d83f10eba\") " pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:59:35.841489 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.839272    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 13:59:35.858888 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.855157    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 13:59:35.865027 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.864999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbnm7\" (UniqueName: \"kubernetes.io/projected/a6752a30-3c25-4979-b28e-b45d83f10eba-kube-api-access-wbnm7\") pod \"network-check-target-5mbt5\" (UID: \"a6752a30-3c25-4979-b28e-b45d83f10eba\") " pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:59:35.871127 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.871107    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9cmv8\""
Sep 08 13:59:35.879389 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.879373    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:59:35.936691 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:35.936650    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-blvrw\" (UID: \"c29aa849-42c2-4f57-9668-4f7435dddec6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:35.936841 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:35.936771    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 13:59:35.936897 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:35.936884    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls podName:c29aa849-42c2-4f57-9668-4f7435dddec6 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:37.936844363 +0000 UTC m=+68.431815038 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-blvrw" (UID: "c29aa849-42c2-4f57-9668-4f7435dddec6") : secret "samples-operator-tls" not found
Sep 08 13:59:36.037752 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:36.037666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:36.037906 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:36.037837    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 13:59:36.037906 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:36.037857    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db7b6f4f5-xv6ht: secret "image-registry-tls" not found
Sep 08 13:59:36.038004 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:36.037932    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls podName:ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab nodeName:}" failed. No retries permitted until 2026-09-08 13:59:38.037916654 +0000 UTC m=+68.532887315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls") pod "image-registry-db7b6f4f5-xv6ht" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab") : secret "image-registry-tls" not found
Sep 08 13:59:36.060205 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:36.060175    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lfxpp"]
Sep 08 13:59:36.063450 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:59:36.063421    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2b891b6_0ff7_4267_b030_ee95f25a0727.slice/crio-48cae7b4be39767e089c95c457e222e131dfd1d5dadc75c71e2891473b0504da WatchSource:0}: Error finding container 48cae7b4be39767e089c95c457e222e131dfd1d5dadc75c71e2891473b0504da: Status 404 returned error can't find the container with id 48cae7b4be39767e089c95c457e222e131dfd1d5dadc75c71e2891473b0504da
Sep 08 13:59:36.162068 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:36.162038    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5mbt5"]
Sep 08 13:59:36.164820 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:59:36.164789    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6752a30_3c25_4979_b28e_b45d83f10eba.slice/crio-512b8111a1e63edf63c93d93b9a5ad7e407415d813b98647edc6219855317e0e WatchSource:0}: Error finding container 512b8111a1e63edf63c93d93b9a5ad7e407415d813b98647edc6219855317e0e: Status 404 returned error can't find the container with id 512b8111a1e63edf63c93d93b9a5ad7e407415d813b98647edc6219855317e0e
Sep 08 13:59:36.219275 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:36.219249    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5mbt5" event={"ID":"a6752a30-3c25-4979-b28e-b45d83f10eba","Type":"ContainerStarted","Data":"512b8111a1e63edf63c93d93b9a5ad7e407415d813b98647edc6219855317e0e"}
Sep 08 13:59:36.220378 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:36.220354    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lfxpp" event={"ID":"c2b891b6-0ff7-4267-b030-ee95f25a0727","Type":"ContainerStarted","Data":"48cae7b4be39767e089c95c457e222e131dfd1d5dadc75c71e2891473b0504da"}
Sep 08 13:59:37.952431 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:37.952392    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-blvrw\" (UID: \"c29aa849-42c2-4f57-9668-4f7435dddec6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:37.952840 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:37.952594    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 13:59:37.952840 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:37.952682    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls podName:c29aa849-42c2-4f57-9668-4f7435dddec6 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:41.952659071 +0000 UTC m=+72.447629736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-blvrw" (UID: "c29aa849-42c2-4f57-9668-4f7435dddec6") : secret "samples-operator-tls" not found
Sep 08 13:59:38.053420 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:38.053373    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:38.053634 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:38.053589    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 13:59:38.053634 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:38.053613    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db7b6f4f5-xv6ht: secret "image-registry-tls" not found
Sep 08 13:59:38.053741 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:38.053695    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls podName:ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab nodeName:}" failed. No retries permitted until 2026-09-08 13:59:42.053672855 +0000 UTC m=+72.548643517 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls") pod "image-registry-db7b6f4f5-xv6ht" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab") : secret "image-registry-tls" not found
Sep 08 13:59:39.379807 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.379689    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n5n7f"]
Sep 08 13:59:39.382225 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.382197    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.385615 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.385596    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 13:59:39.385757 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.385675    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 13:59:39.386097 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.386075    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 13:59:39.386782 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.386725    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8q4kl\""
Sep 08 13:59:39.391648 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.391630    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 13:59:39.407076 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.407000    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n5n7f"]
Sep 08 13:59:39.465422 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.465391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.465580 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.465462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.465580 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.465521    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c39d180-d8f0-4b18-b7ff-70563ea2e867-data-volume\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.465580 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.465575    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndjt8\" (UniqueName: \"kubernetes.io/projected/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-api-access-ndjt8\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.465760 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.465624    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c39d180-d8f0-4b18-b7ff-70563ea2e867-crio-socket\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.566159 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.566113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndjt8\" (UniqueName: \"kubernetes.io/projected/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-api-access-ndjt8\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.566340 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.566183    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c39d180-d8f0-4b18-b7ff-70563ea2e867-crio-socket\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.566340 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.566210    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.566340 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.566238    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.566340 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.566288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c39d180-d8f0-4b18-b7ff-70563ea2e867-data-volume\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.566340 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.566323    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7c39d180-d8f0-4b18-b7ff-70563ea2e867-crio-socket\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.566536 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:39.566341    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:59:39.566536 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:39.566406    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:39.566536 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:39.566431    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm podName:7c39d180-d8f0-4b18-b7ff-70563ea2e867 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:40.06640929 +0000 UTC m=+70.561379967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n5n7f" (UID: "7c39d180-d8f0-4b18-b7ff-70563ea2e867") : configmap "kube-rbac-proxy" not found
Sep 08 13:59:39.566536 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:39.566469    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls podName:7c39d180-d8f0-4b18-b7ff-70563ea2e867 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:40.0664515 +0000 UTC m=+70.561422177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5n7f" (UID: "7c39d180-d8f0-4b18-b7ff-70563ea2e867") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:39.566662 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.566612    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c39d180-d8f0-4b18-b7ff-70563ea2e867-data-volume\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:39.578205 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:39.578171    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndjt8\" (UniqueName: \"kubernetes.io/projected/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-api-access-ndjt8\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:40.070024 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:40.069983    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:40.070188 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:40.070037    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:40.070188 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:40.070127    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:59:40.070188 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:40.070166    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:40.070289 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:40.070198    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm podName:7c39d180-d8f0-4b18-b7ff-70563ea2e867 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:41.070180848 +0000 UTC m=+71.565151510 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n5n7f" (UID: "7c39d180-d8f0-4b18-b7ff-70563ea2e867") : configmap "kube-rbac-proxy" not found
Sep 08 13:59:40.070289 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:40.070213    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls podName:7c39d180-d8f0-4b18-b7ff-70563ea2e867 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:41.070207232 +0000 UTC m=+71.565177894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5n7f" (UID: "7c39d180-d8f0-4b18-b7ff-70563ea2e867") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:40.232828 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:40.232791    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/0.log"
Sep 08 13:59:40.232975 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:40.232841    2564 generic.go:356] "Generic (PLEG): container finished" podID="7c2f5dad-d885-4865-84ba-9a31338284b5" containerID="abbd15e9d9988ef8f8f4af106e1fe06349a13f8a01ccea9ee715edd6062171af" exitCode=255
Sep 08 13:59:40.232975 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:40.232893    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-7s6g6" event={"ID":"7c2f5dad-d885-4865-84ba-9a31338284b5","Type":"ContainerDied","Data":"abbd15e9d9988ef8f8f4af106e1fe06349a13f8a01ccea9ee715edd6062171af"}
Sep 08 13:59:40.233226 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:40.233211    2564 scope.go:117] "RemoveContainer" containerID="abbd15e9d9988ef8f8f4af106e1fe06349a13f8a01ccea9ee715edd6062171af"
Sep 08 13:59:40.234551 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:40.234525    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5mbt5" event={"ID":"a6752a30-3c25-4979-b28e-b45d83f10eba","Type":"ContainerStarted","Data":"29923a6b23f59afee3bb3937ca6314a76dc66b54776455d255881768dd6cddcf"}
Sep 08 13:59:40.234626 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:40.234586    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 13:59:40.266598 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:40.266553    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5mbt5" podStartSLOduration=66.743853348 podStartE2EDuration="1m10.266537134s" totalImagesPullingTime="3.522683786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:59:40.266265612 +0000 UTC m=+70.761236297" watchObservedRunningTime="2026-09-08 13:59:40.266537134 +0000 UTC m=+70.761507818"
Sep 08 13:59:41.067667 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.067636    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qgvf6_dbfe671c-f26a-484b-8e4a-d634b1731f98/dns-node-resolver/0.log"
Sep 08 13:59:41.078971 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.078935    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:41.079133 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.079061    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:41.079133 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:41.079092    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:41.079133 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:41.079130    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:59:41.079292 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:41.079157    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls podName:7c39d180-d8f0-4b18-b7ff-70563ea2e867 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:43.079139568 +0000 UTC m=+73.574110247 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5n7f" (UID: "7c39d180-d8f0-4b18-b7ff-70563ea2e867") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:41.079292 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:41.079175    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm podName:7c39d180-d8f0-4b18-b7ff-70563ea2e867 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:43.079165662 +0000 UTC m=+73.574136326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n5n7f" (UID: "7c39d180-d8f0-4b18-b7ff-70563ea2e867") : configmap "kube-rbac-proxy" not found
Sep 08 13:59:41.239994 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.239962    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 13:59:41.240475 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.240458    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/0.log"
Sep 08 13:59:41.240567 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.240499    2564 generic.go:356] "Generic (PLEG): container finished" podID="7c2f5dad-d885-4865-84ba-9a31338284b5" containerID="a2214000e959eb5ccef8cb97d034d7b594b5ceb1d68eff8aae459586a07c0c0c" exitCode=255
Sep 08 13:59:41.240567 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.240535    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-7s6g6" event={"ID":"7c2f5dad-d885-4865-84ba-9a31338284b5","Type":"ContainerDied","Data":"a2214000e959eb5ccef8cb97d034d7b594b5ceb1d68eff8aae459586a07c0c0c"}
Sep 08 13:59:41.240672 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.240607    2564 scope.go:117] "RemoveContainer" containerID="abbd15e9d9988ef8f8f4af106e1fe06349a13f8a01ccea9ee715edd6062171af"
Sep 08 13:59:41.240844 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.240827    2564 scope.go:117] "RemoveContainer" containerID="a2214000e959eb5ccef8cb97d034d7b594b5ceb1d68eff8aae459586a07c0c0c"
Sep 08 13:59:41.241076 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:41.241047    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-7s6g6_openshift-console-operator(7c2f5dad-d885-4865-84ba-9a31338284b5)\"" pod="openshift-console-operator/console-operator-85b454874b-7s6g6" podUID="7c2f5dad-d885-4865-84ba-9a31338284b5"
Sep 08 13:59:41.384734 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.384656    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-4q4kx"]
Sep 08 13:59:41.387531 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.387506    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-4q4kx"
Sep 08 13:59:41.390266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.390003    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 08 13:59:41.390266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.390100    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hfbgn\""
Sep 08 13:59:41.390266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.390108    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 08 13:59:41.390266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.390175    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 08 13:59:41.390266 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.390187    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 08 13:59:41.398318 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.398261    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-4q4kx"]
Sep 08 13:59:41.482501 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.482466    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/29a0517a-6361-4801-a4ef-0bcc337d4d80-signing-key\") pod \"service-ca-75d99c484-4q4kx\" (UID: \"29a0517a-6361-4801-a4ef-0bcc337d4d80\") " pod="openshift-service-ca/service-ca-75d99c484-4q4kx"
Sep 08 13:59:41.482701 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.482519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/29a0517a-6361-4801-a4ef-0bcc337d4d80-signing-cabundle\") pod \"service-ca-75d99c484-4q4kx\" (UID: \"29a0517a-6361-4801-a4ef-0bcc337d4d80\") " pod="openshift-service-ca/service-ca-75d99c484-4q4kx"
Sep 08 13:59:41.482701 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.482554    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hrgc\" (UniqueName: \"kubernetes.io/projected/29a0517a-6361-4801-a4ef-0bcc337d4d80-kube-api-access-7hrgc\") pod \"service-ca-75d99c484-4q4kx\" (UID: \"29a0517a-6361-4801-a4ef-0bcc337d4d80\") " pod="openshift-service-ca/service-ca-75d99c484-4q4kx"
Sep 08 13:59:41.583203 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.583170    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/29a0517a-6361-4801-a4ef-0bcc337d4d80-signing-key\") pod \"service-ca-75d99c484-4q4kx\" (UID: \"29a0517a-6361-4801-a4ef-0bcc337d4d80\") " pod="openshift-service-ca/service-ca-75d99c484-4q4kx"
Sep 08 13:59:41.583474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.583216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/29a0517a-6361-4801-a4ef-0bcc337d4d80-signing-cabundle\") pod \"service-ca-75d99c484-4q4kx\" (UID: \"29a0517a-6361-4801-a4ef-0bcc337d4d80\") " pod="openshift-service-ca/service-ca-75d99c484-4q4kx"
Sep 08 13:59:41.583474 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.583245    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hrgc\" (UniqueName: \"kubernetes.io/projected/29a0517a-6361-4801-a4ef-0bcc337d4d80-kube-api-access-7hrgc\") pod \"service-ca-75d99c484-4q4kx\" (UID: \"29a0517a-6361-4801-a4ef-0bcc337d4d80\") " pod="openshift-service-ca/service-ca-75d99c484-4q4kx"
Sep 08 13:59:41.584109 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.584081    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/29a0517a-6361-4801-a4ef-0bcc337d4d80-signing-cabundle\") pod \"service-ca-75d99c484-4q4kx\" (UID: \"29a0517a-6361-4801-a4ef-0bcc337d4d80\") " pod="openshift-service-ca/service-ca-75d99c484-4q4kx"
Sep 08 13:59:41.586173 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.586148    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/29a0517a-6361-4801-a4ef-0bcc337d4d80-signing-key\") pod \"service-ca-75d99c484-4q4kx\" (UID: \"29a0517a-6361-4801-a4ef-0bcc337d4d80\") " pod="openshift-service-ca/service-ca-75d99c484-4q4kx"
Sep 08 13:59:41.592909 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.592887    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hrgc\" (UniqueName: \"kubernetes.io/projected/29a0517a-6361-4801-a4ef-0bcc337d4d80-kube-api-access-7hrgc\") pod \"service-ca-75d99c484-4q4kx\" (UID: \"29a0517a-6361-4801-a4ef-0bcc337d4d80\") " pod="openshift-service-ca/service-ca-75d99c484-4q4kx"
Sep 08 13:59:41.699070 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.698968    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-4q4kx"
Sep 08 13:59:41.986532 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:41.986500    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-blvrw\" (UID: \"c29aa849-42c2-4f57-9668-4f7435dddec6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:41.986697 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:41.986646    2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 13:59:41.986742 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:41.986705    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls podName:c29aa849-42c2-4f57-9668-4f7435dddec6 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:49.98668852 +0000 UTC m=+80.481659184 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-blvrw" (UID: "c29aa849-42c2-4f57-9668-4f7435dddec6") : secret "samples-operator-tls" not found
Sep 08 13:59:42.069404 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:42.069377    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-47jpr_da52533e-1455-4ff1-8942-6cf7633a85a1/node-ca/0.log"
Sep 08 13:59:42.087877 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:42.087829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:42.088046 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:42.087978    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 13:59:42.088046 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:42.087998    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-db7b6f4f5-xv6ht: secret "image-registry-tls" not found
Sep 08 13:59:42.088152 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:42.088067    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls podName:ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab nodeName:}" failed. No retries permitted until 2026-09-08 13:59:50.088046094 +0000 UTC m=+80.583016963 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls") pod "image-registry-db7b6f4f5-xv6ht" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab") : secret "image-registry-tls" not found
Sep 08 13:59:42.244022 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:42.243934    2564 scope.go:117] "RemoveContainer" containerID="a2214000e959eb5ccef8cb97d034d7b594b5ceb1d68eff8aae459586a07c0c0c"
Sep 08 13:59:42.244175 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:42.244135    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-7s6g6_openshift-console-operator(7c2f5dad-d885-4865-84ba-9a31338284b5)\"" pod="openshift-console-operator/console-operator-85b454874b-7s6g6" podUID="7c2f5dad-d885-4865-84ba-9a31338284b5"
Sep 08 13:59:42.565081 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:42.565000    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-4q4kx"]
Sep 08 13:59:42.568600 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:59:42.568575    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a0517a_6361_4801_a4ef_0bcc337d4d80.slice/crio-25ee3a642b7470ef752d0f15fdaaf231a4e76807bd9d0ccb80ca047b0d092218 WatchSource:0}: Error finding container 25ee3a642b7470ef752d0f15fdaaf231a4e76807bd9d0ccb80ca047b0d092218: Status 404 returned error can't find the container with id 25ee3a642b7470ef752d0f15fdaaf231a4e76807bd9d0ccb80ca047b0d092218
Sep 08 13:59:43.067424 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:43.067393    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qsrr6_d4066c24-5beb-4011-a94b-cf594601f554/serve-healthcheck-canary/0.log"
Sep 08 13:59:43.095090 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:43.095051    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:43.095496 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:43.095119    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:43.095496 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:43.095203    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:59:43.095496 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:43.095252    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:43.095496 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:43.095298    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm podName:7c39d180-d8f0-4b18-b7ff-70563ea2e867 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:47.09527839 +0000 UTC m=+77.590249055 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n5n7f" (UID: "7c39d180-d8f0-4b18-b7ff-70563ea2e867") : configmap "kube-rbac-proxy" not found
Sep 08 13:59:43.095496 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:43.095314    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls podName:7c39d180-d8f0-4b18-b7ff-70563ea2e867 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:47.095308395 +0000 UTC m=+77.590279056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5n7f" (UID: "7c39d180-d8f0-4b18-b7ff-70563ea2e867") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:43.247706 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:43.247663    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-4q4kx" event={"ID":"29a0517a-6361-4801-a4ef-0bcc337d4d80","Type":"ContainerStarted","Data":"25ee3a642b7470ef752d0f15fdaaf231a4e76807bd9d0ccb80ca047b0d092218"}
Sep 08 13:59:43.249497 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:43.249466    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lfxpp" event={"ID":"c2b891b6-0ff7-4267-b030-ee95f25a0727","Type":"ContainerStarted","Data":"2fe6db574f9ee2fc3baf6cd1f6ba8f0591e01042a3dba9a26413104a762faa56"}
Sep 08 13:59:43.251236 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:43.251207    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 13:59:43.266082 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:43.266035    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lfxpp" podStartSLOduration=66.968087363 podStartE2EDuration="1m13.266019342s" totalImagesPullingTime="6.297931979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:59:43.265362101 +0000 UTC m=+73.760332786" watchObservedRunningTime="2026-09-08 13:59:43.266019342 +0000 UTC m=+73.760990027"
Sep 08 13:59:44.681304 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:44.681193    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:44.681304 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:44.681269    2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:44.681827 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:44.681690    2564 scope.go:117] "RemoveContainer" containerID="a2214000e959eb5ccef8cb97d034d7b594b5ceb1d68eff8aae459586a07c0c0c"
Sep 08 13:59:44.681944 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:44.681918    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-7s6g6_openshift-console-operator(7c2f5dad-d885-4865-84ba-9a31338284b5)\"" pod="openshift-console-operator/console-operator-85b454874b-7s6g6" podUID="7c2f5dad-d885-4865-84ba-9a31338284b5"
Sep 08 13:59:45.258503 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:45.258467    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-4q4kx" event={"ID":"29a0517a-6361-4801-a4ef-0bcc337d4d80","Type":"ContainerStarted","Data":"045ba401c0c0a5e8b240cccd20bf49411dd53f8430fc97d2e351e3d928f0e17c"}
Sep 08 13:59:45.274774 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:45.274732    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-4q4kx" podStartSLOduration=2.454603848 podStartE2EDuration="4.274717322s" totalImagesPullingTime="1.820113474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:59:45.274651444 +0000 UTC m=+75.769622129" watchObservedRunningTime="2026-09-08 13:59:45.274717322 +0000 UTC m=+75.769688006"
Sep 08 13:59:47.127772 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:47.127740    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:47.128145 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:47.127828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:47.128145 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:47.127919    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:47.128145 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:47.127991    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls podName:7c39d180-d8f0-4b18-b7ff-70563ea2e867 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:55.127969167 +0000 UTC m=+85.622939832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5n7f" (UID: "7c39d180-d8f0-4b18-b7ff-70563ea2e867") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:47.128145 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:47.127922    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:59:47.128145 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:47.128042    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm podName:7c39d180-d8f0-4b18-b7ff-70563ea2e867 nodeName:}" failed. No retries permitted until 2026-09-08 13:59:55.128029113 +0000 UTC m=+85.622999775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n5n7f" (UID: "7c39d180-d8f0-4b18-b7ff-70563ea2e867") : configmap "kube-rbac-proxy" not found
Sep 08 13:59:50.046161 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:50.046127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-blvrw\" (UID: \"c29aa849-42c2-4f57-9668-4f7435dddec6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:50.048836 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:50.048801    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c29aa849-42c2-4f57-9668-4f7435dddec6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-blvrw\" (UID: \"c29aa849-42c2-4f57-9668-4f7435dddec6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:50.147355 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:50.147318    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:50.149823 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:50.149803    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls\") pod \"image-registry-db7b6f4f5-xv6ht\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") " pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:50.151663 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:50.151641    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"
Sep 08 13:59:50.280470 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:50.280444    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:50.420506 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:50.420476    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw"]
Sep 08 13:59:50.552771 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:50.552741    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db7b6f4f5-xv6ht"]
Sep 08 13:59:50.555965 ip-10-0-133-79 kubenswrapper[2564]: W0908 13:59:50.555937    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab3b46b5_1b4c_4393_8554_ebbc9f3a09ab.slice/crio-2ec769335c9586728ae0dc160d85b0fd3199e20e154ed2f816b4c7587c8052b9 WatchSource:0}: Error finding container 2ec769335c9586728ae0dc160d85b0fd3199e20e154ed2f816b4c7587c8052b9: Status 404 returned error can't find the container with id 2ec769335c9586728ae0dc160d85b0fd3199e20e154ed2f816b4c7587c8052b9
Sep 08 13:59:51.276273 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:51.276240    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht" event={"ID":"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab","Type":"ContainerStarted","Data":"742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078"}
Sep 08 13:59:51.276734 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:51.276299    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht" event={"ID":"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab","Type":"ContainerStarted","Data":"2ec769335c9586728ae0dc160d85b0fd3199e20e154ed2f816b4c7587c8052b9"}
Sep 08 13:59:51.276734 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:51.276353    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 13:59:51.277892 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:51.277848    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw" event={"ID":"c29aa849-42c2-4f57-9668-4f7435dddec6","Type":"ContainerStarted","Data":"491fddc54079a58d99d577c3750a4f79ae80efa3ec68860d7c73b05ab68f7ad1"}
Sep 08 13:59:51.297854 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:51.297799    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht" podStartSLOduration=17.297777892 podStartE2EDuration="17.297777892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:59:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:59:51.295711986 +0000 UTC m=+81.790682670" watchObservedRunningTime="2026-09-08 13:59:51.297777892 +0000 UTC m=+81.792748577"
Sep 08 13:59:52.284087 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:52.283991    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw" event={"ID":"c29aa849-42c2-4f57-9668-4f7435dddec6","Type":"ContainerStarted","Data":"4c640ecf5608f9dbb16d0b1898e64512b64de4cf789691e6e937277389c7a92a"}
Sep 08 13:59:52.284499 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:52.284103    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw" event={"ID":"c29aa849-42c2-4f57-9668-4f7435dddec6","Type":"ContainerStarted","Data":"4702477ff960b4b6126562ddc299b5bda17697b1b5787ca7e3798b341603bb02"}
Sep 08 13:59:52.301761 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:52.301715    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-blvrw" podStartSLOduration=16.767204565 podStartE2EDuration="18.301700982s" totalImagesPullingTime="1.534496417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:59:52.300842267 +0000 UTC m=+82.795812932" watchObservedRunningTime="2026-09-08 13:59:52.301700982 +0000 UTC m=+82.796671665"
Sep 08 13:59:55.192180 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:55.192139    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:55.192180 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:55.192190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:55.192741 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:55.192303    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:55.192741 ip-10-0-133-79 kubenswrapper[2564]: E0908 13:59:55.192364    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls podName:7c39d180-d8f0-4b18-b7ff-70563ea2e867 nodeName:}" failed. No retries permitted until 2026-09-08 14:00:11.192349888 +0000 UTC m=+101.687320549 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5n7f" (UID: "7c39d180-d8f0-4b18-b7ff-70563ea2e867") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:59:55.192901 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:55.192810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7c39d180-d8f0-4b18-b7ff-70563ea2e867-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 13:59:55.952675 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:55.952642    2564 scope.go:117] "RemoveContainer" containerID="a2214000e959eb5ccef8cb97d034d7b594b5ceb1d68eff8aae459586a07c0c0c"
Sep 08 13:59:56.303160 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:56.303132    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 13:59:56.303559 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:56.303199    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-7s6g6" event={"ID":"7c2f5dad-d885-4865-84ba-9a31338284b5","Type":"ContainerStarted","Data":"1b52097c3ac3e45ca0edcf5575165187eacf4c6c5fffb28d33670fd7d1c7d3d0"}
Sep 08 13:59:56.303559 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:56.303496    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 13:59:56.323985 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:56.323934    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-7s6g6" podStartSLOduration=17.635606038 podStartE2EDuration="22.323916964s" totalImagesPullingTime="4.688310926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:59:56.32280773 +0000 UTC m=+86.817778414" watchObservedRunningTime="2026-09-08 13:59:56.323916964 +0000 UTC m=+86.818887650"
Sep 08 13:59:56.555280 ip-10-0-133-79 kubenswrapper[2564]: I0908 13:59:56.555200    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-7s6g6"
Sep 08 14:00:02.808229 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.808198    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"]
Sep 08 14:00:02.811576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.811558    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:02.816703 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.816677    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 08 14:00:02.817732 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.817715    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 08 14:00:02.817924 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.817838    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 08 14:00:02.818052 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.818032    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 08 14:00:02.818139 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.818043    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 08 14:00:02.818199 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.818129    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 08 14:00:02.818801 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.818780    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 08 14:00:02.841134 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.841106    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"]
Sep 08 14:00:02.894450 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.894415    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-db7b6f4f5-xv6ht"]
Sep 08 14:00:02.949294 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.949262    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:02.949294 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.949294    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-hub\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:02.949505 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.949315    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-ca\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:02.949505 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.949332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8df25df-a7f0-493f-b156-6aacd11d3af9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:02.949505 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.949416    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:02.949505 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:02.949431    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cttv6\" (UniqueName: \"kubernetes.io/projected/b8df25df-a7f0-493f-b156-6aacd11d3af9-kube-api-access-cttv6\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.050640 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.050604    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.050640 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.050639    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-hub\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.050934 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.050656    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-ca\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.050934 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.050671    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8df25df-a7f0-493f-b156-6aacd11d3af9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.050934 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.050732    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.050934 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.050758    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cttv6\" (UniqueName: \"kubernetes.io/projected/b8df25df-a7f0-493f-b156-6aacd11d3af9-kube-api-access-cttv6\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.051475 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.051445    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8df25df-a7f0-493f-b156-6aacd11d3af9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.053279 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.053246    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-ca\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.053392 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.053349    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.053756 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.053738    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.053839 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.053819    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b8df25df-a7f0-493f-b156-6aacd11d3af9-hub\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.064575 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.064519    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cttv6\" (UniqueName: \"kubernetes.io/projected/b8df25df-a7f0-493f-b156-6aacd11d3af9-kube-api-access-cttv6\") pod \"cluster-proxy-proxy-agent-7898f955cf-688ch\" (UID: \"b8df25df-a7f0-493f-b156-6aacd11d3af9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.128457 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.128422    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"
Sep 08 14:00:03.437089 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:03.437066    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch"]
Sep 08 14:00:03.439457 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:00:03.439433    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8df25df_a7f0_493f_b156_6aacd11d3af9.slice/crio-9aefc25039c7ae5e98b7f83b72170c52720ec1ed6124c92da3c15314dfc59c1a WatchSource:0}: Error finding container 9aefc25039c7ae5e98b7f83b72170c52720ec1ed6124c92da3c15314dfc59c1a: Status 404 returned error can't find the container with id 9aefc25039c7ae5e98b7f83b72170c52720ec1ed6124c92da3c15314dfc59c1a
Sep 08 14:00:04.329540 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:04.329501    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch" event={"ID":"b8df25df-a7f0-493f-b156-6aacd11d3af9","Type":"ContainerStarted","Data":"9aefc25039c7ae5e98b7f83b72170c52720ec1ed6124c92da3c15314dfc59c1a"}
Sep 08 14:00:07.795070 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:07.791571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 14:00:07.795070 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:07.794703    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a313f4b1-b29f-4d5f-a580-5ed5bd99696d-metrics-tls\") pod \"dns-default-fwk9m\" (UID: \"a313f4b1-b29f-4d5f-a580-5ed5bd99696d\") " pod="openshift-dns/dns-default-fwk9m"
Sep 08 14:00:07.817555 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:07.817519    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b4mdc\""
Sep 08 14:00:07.825792 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:07.825761    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fwk9m"
Sep 08 14:00:08.707417 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:08.707338    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fwk9m"]
Sep 08 14:00:08.711537 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:00:08.711501    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda313f4b1_b29f_4d5f_a580_5ed5bd99696d.slice/crio-a11dcadde64bfeda331e4dfbe970a5d3743bcef82f13b34d4e9bee77934df4fd WatchSource:0}: Error finding container a11dcadde64bfeda331e4dfbe970a5d3743bcef82f13b34d4e9bee77934df4fd: Status 404 returned error can't find the container with id a11dcadde64bfeda331e4dfbe970a5d3743bcef82f13b34d4e9bee77934df4fd
Sep 08 14:00:09.352169 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:09.352123    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fwk9m" event={"ID":"a313f4b1-b29f-4d5f-a580-5ed5bd99696d","Type":"ContainerStarted","Data":"a11dcadde64bfeda331e4dfbe970a5d3743bcef82f13b34d4e9bee77934df4fd"}
Sep 08 14:00:11.221257 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:11.221220    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 14:00:11.223716 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:11.223694    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7c39d180-d8f0-4b18-b7ff-70563ea2e867-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5n7f\" (UID: \"7c39d180-d8f0-4b18-b7ff-70563ea2e867\") " pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 14:00:11.243565 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:11.243540    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5mbt5"
Sep 08 14:00:11.361887 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:11.361824    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch" event={"ID":"b8df25df-a7f0-493f-b156-6aacd11d3af9","Type":"ContainerStarted","Data":"ae8a9b400444d69e71952595c9b4a7ef42671ac612286fe395ddb53ffa65ca50"}
Sep 08 14:00:11.362219 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:11.361895    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch" event={"ID":"b8df25df-a7f0-493f-b156-6aacd11d3af9","Type":"ContainerStarted","Data":"d4541aef8c901c3163c38e04a724d2c2e16ac0dc676b41ec38d202eefebbee9f"}
Sep 08 14:00:11.362219 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:11.361916    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch" event={"ID":"b8df25df-a7f0-493f-b156-6aacd11d3af9","Type":"ContainerStarted","Data":"d8db00de4ab760f97bdeb3a7bd68a8f743b161f852c3bf2ee68296f400a0e2e9"}
Sep 08 14:00:11.381765 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:11.381712    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7898f955cf-688ch" podStartSLOduration=2.288848837 podStartE2EDuration="9.381695063s" totalImagesPullingTime="7.092846226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:00:11.381085704 +0000 UTC m=+101.876056388" watchObservedRunningTime="2026-09-08 14:00:11.381695063 +0000 UTC m=+101.876665746"
Sep 08 14:00:11.494718 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:11.494622    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n5n7f"
Sep 08 14:00:11.770686 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:11.770585    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n5n7f"]
Sep 08 14:00:11.776701 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:00:11.776662    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c39d180_d8f0_4b18_b7ff_70563ea2e867.slice/crio-76c8db8073538db2f207ba231f1bd5d90291b5e779c75958ad093113b6c72595 WatchSource:0}: Error finding container 76c8db8073538db2f207ba231f1bd5d90291b5e779c75958ad093113b6c72595: Status 404 returned error can't find the container with id 76c8db8073538db2f207ba231f1bd5d90291b5e779c75958ad093113b6c72595
Sep 08 14:00:12.367951 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:12.367910    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5n7f" event={"ID":"7c39d180-d8f0-4b18-b7ff-70563ea2e867","Type":"ContainerStarted","Data":"acd4e592a2472b19ed674c37783f281afdc3095ff27666e8c006649cc9645a84"}
Sep 08 14:00:12.368429 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:12.367984    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5n7f" event={"ID":"7c39d180-d8f0-4b18-b7ff-70563ea2e867","Type":"ContainerStarted","Data":"76c8db8073538db2f207ba231f1bd5d90291b5e779c75958ad093113b6c72595"}
Sep 08 14:00:12.902387 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:12.902356    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 14:00:14.379965 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:14.379926    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5n7f" event={"ID":"7c39d180-d8f0-4b18-b7ff-70563ea2e867","Type":"ContainerStarted","Data":"ba6c39bb6d3fb1de5c0ce507a9cc8a7859e9d76d56c52b9249cfde4ccb78a7c1"}
Sep 08 14:00:14.381634 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:14.381605    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fwk9m" event={"ID":"a313f4b1-b29f-4d5f-a580-5ed5bd99696d","Type":"ContainerStarted","Data":"850bde7256fc4302b6b30521c1c1175953fc783e3e53a99122a3b20f8089544b"}
Sep 08 14:00:14.381764 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:14.381649    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fwk9m" event={"ID":"a313f4b1-b29f-4d5f-a580-5ed5bd99696d","Type":"ContainerStarted","Data":"239987a3a35b333a286ce904254c90e3385f9dca872258cb2d6caeb4283659f5"}
Sep 08 14:00:14.381829 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:14.381763    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fwk9m"
Sep 08 14:00:14.400763 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:14.400721    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fwk9m" podStartSLOduration=66.5843377 podStartE2EDuration="1m11.400706857s" totalImagesPullingTime="4.816369157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:00:14.399818894 +0000 UTC m=+104.894789578" watchObservedRunningTime="2026-09-08 14:00:14.400706857 +0000 UTC m=+104.895677541"
Sep 08 14:00:17.396218 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:17.396181    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5n7f" event={"ID":"7c39d180-d8f0-4b18-b7ff-70563ea2e867","Type":"ContainerStarted","Data":"948add4bff2c18df4354bb89767036c024060f41852716b210fb0993611fbe70"}
Sep 08 14:00:17.417551 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:17.417494    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n5n7f" podStartSLOduration=33.45775135 podStartE2EDuration="38.417478811s" totalImagesPullingTime="4.959727461s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:59:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:00:17.416980169 +0000 UTC m=+107.911950853" watchObservedRunningTime="2026-09-08 14:00:17.417478811 +0000 UTC m=+107.912449494"
Sep 08 14:00:24.388277 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:24.388248    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fwk9m"
Sep 08 14:00:27.914107 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:27.914063    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht" podUID="ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" containerName="registry" containerID="cri-o://742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078" gracePeriod=30
Sep 08 14:00:28.145677 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.145655    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 14:00:28.253883 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.253830    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls\") pod \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") "
Sep 08 14:00:28.254027 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.253910    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-ca-trust-extracted\") pod \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") "
Sep 08 14:00:28.254027 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.253950    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-trusted-ca\") pod \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") "
Sep 08 14:00:28.254027 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.253968    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-installation-pull-secrets\") pod \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") "
Sep 08 14:00:28.254027 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.254004    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-image-registry-private-configuration\") pod \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") "
Sep 08 14:00:28.254232 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.254159    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-certificates\") pod \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") "
Sep 08 14:00:28.254278 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.254230    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-bound-sa-token\") pod \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") "
Sep 08 14:00:28.254329 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.254282    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7xzq\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-kube-api-access-n7xzq\") pod \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\" (UID: \"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab\") "
Sep 08 14:00:28.254555 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.254521    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:00:28.254555 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.254540    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:00:28.256534 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.256502    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:00:28.256625 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.256586    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:00:28.256779 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.256748    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:00:28.257236 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.257216    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:00:28.257297 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.257215    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-kube-api-access-n7xzq" (OuterVolumeSpecName: "kube-api-access-n7xzq") pod "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab"). InnerVolumeSpecName "kube-api-access-n7xzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:00:28.262606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.262584    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" (UID: "ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:00:28.355378 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.355342    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-bound-sa-token\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:00:28.355378 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.355370    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7xzq\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-kube-api-access-n7xzq\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:00:28.355378 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.355380    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-tls\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:00:28.355586 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.355390    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-ca-trust-extracted\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:00:28.355586 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.355399    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-trusted-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:00:28.355586 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.355408    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-installation-pull-secrets\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:00:28.355586 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.355417    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-image-registry-private-configuration\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:00:28.355586 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.355426    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab-registry-certificates\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:00:28.438012 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.437977    2564 generic.go:356] "Generic (PLEG): container finished" podID="ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" containerID="742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078" exitCode=0
Sep 08 14:00:28.438171 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.438035    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht"
Sep 08 14:00:28.438171 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.438063    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht" event={"ID":"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab","Type":"ContainerDied","Data":"742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078"}
Sep 08 14:00:28.438171 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.438136    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db7b6f4f5-xv6ht" event={"ID":"ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab","Type":"ContainerDied","Data":"2ec769335c9586728ae0dc160d85b0fd3199e20e154ed2f816b4c7587c8052b9"}
Sep 08 14:00:28.438171 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.438157    2564 scope.go:117] "RemoveContainer" containerID="742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078"
Sep 08 14:00:28.447080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.447066    2564 scope.go:117] "RemoveContainer" containerID="742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078"
Sep 08 14:00:28.447388 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:00:28.447327    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078\": container with ID starting with 742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078 not found: ID does not exist" containerID="742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078"
Sep 08 14:00:28.447433 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.447396    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078"} err="failed to get container status \"742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078\": rpc error: code = NotFound desc = could not find container \"742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078\": container with ID starting with 742e7d836019abf3af5b19404773e7873d938154aacb4de27ad23e634d683078 not found: ID does not exist"
Sep 08 14:00:28.466991 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.466965    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-db7b6f4f5-xv6ht"]
Sep 08 14:00:28.475400 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.475365    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-db7b6f4f5-xv6ht"]
Sep 08 14:00:28.622764 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.622675    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"]
Sep 08 14:00:28.622953 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.622937    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" containerName="registry"
Sep 08 14:00:28.622953 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.622949    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" containerName="registry"
Sep 08 14:00:28.623062 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.622995    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" containerName="registry"
Sep 08 14:00:28.626204 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.626186    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.630063 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.630039    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 08 14:00:28.630183 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.630079    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 14:00:28.630183 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.630093    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 14:00:28.630381 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.630365    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 14:00:28.630707 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.630688    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 08 14:00:28.631257 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.631240    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rt9c4\""
Sep 08 14:00:28.638947 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.638854    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"]
Sep 08 14:00:28.671333 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.671302    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"]
Sep 08 14:00:28.673285 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.673269    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.676680 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.676646    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 08 14:00:28.676680 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.676665    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rhbmg\""
Sep 08 14:00:28.676946 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.676931    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 08 14:00:28.677043 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.676966    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 08 14:00:28.679934 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.679896    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fr5t8"]
Sep 08 14:00:28.681725 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.681708    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.684226 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.684207    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 14:00:28.684226 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.684221    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-phh7g\""
Sep 08 14:00:28.684403 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.684210    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 14:00:28.684403 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.684279    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 14:00:28.689667 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.689646    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"]
Sep 08 14:00:28.762880 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.762824    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d9c7549-ca97-450c-9de8-9caf01336dbf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.763073 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.762899    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d9c7549-ca97-450c-9de8-9caf01336dbf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.763073 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.762963    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wjw9\" (UniqueName: \"kubernetes.io/projected/0d9c7549-ca97-450c-9de8-9caf01336dbf-kube-api-access-2wjw9\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.763073 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.763048    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d9c7549-ca97-450c-9de8-9caf01336dbf-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.864182 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864149    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wjw9\" (UniqueName: \"kubernetes.io/projected/0d9c7549-ca97-450c-9de8-9caf01336dbf-kube-api-access-2wjw9\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.864182 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864199    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d9c7549-ca97-450c-9de8-9caf01336dbf-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.864430 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864229    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-tls\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.864430 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864255    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.864430 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864277    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-sys\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.864430 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864301    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-root\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.864430 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864328    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.864430 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864378    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7sln\" (UniqueName: \"kubernetes.io/projected/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-api-access-g7sln\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.864430 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864410    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.864777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d9c7549-ca97-450c-9de8-9caf01336dbf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.864777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864464    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-metrics-client-ca\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.864777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864495    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.864777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864525    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.864777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864551    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-wtmp\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.864777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d9c7549-ca97-450c-9de8-9caf01336dbf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.864777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864620    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.864777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864666    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.864777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864710    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkktd\" (UniqueName: \"kubernetes.io/projected/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-kube-api-access-qkktd\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.864777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.864746    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-textfile\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.865894 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.865848    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d9c7549-ca97-450c-9de8-9caf01336dbf-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.866018 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:00:28.865965    2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 08 14:00:28.866079 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:00:28.866032    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d9c7549-ca97-450c-9de8-9caf01336dbf-openshift-state-metrics-tls podName:0d9c7549-ca97-450c-9de8-9caf01336dbf nodeName:}" failed. No retries permitted until 2026-09-08 14:00:29.366012852 +0000 UTC m=+119.860983532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/0d9c7549-ca97-450c-9de8-9caf01336dbf-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-22mzn" (UID: "0d9c7549-ca97-450c-9de8-9caf01336dbf") : secret "openshift-state-metrics-tls" not found
Sep 08 14:00:28.868414 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.868385    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d9c7549-ca97-450c-9de8-9caf01336dbf-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.873833 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.873767    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wjw9\" (UniqueName: \"kubernetes.io/projected/0d9c7549-ca97-450c-9de8-9caf01336dbf-kube-api-access-2wjw9\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:28.965149 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965114    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.965595 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkktd\" (UniqueName: \"kubernetes.io/projected/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-kube-api-access-qkktd\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.965595 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965310    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-textfile\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.965595 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-tls\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.965595 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965401    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.965595 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965442    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-sys\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.965595 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965479    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-root\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.965595 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.965595 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965560    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7sln\" (UniqueName: \"kubernetes.io/projected/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-api-access-g7sln\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.965595 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.966080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965624    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-metrics-client-ca\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.966080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965658    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.966080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965687    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.966080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965716    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-wtmp\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.966080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.966080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965782    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-textfile\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.966080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.965950    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.966080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.966024    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-root\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.966080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.966068    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-sys\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.966523 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.966086    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.966523 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.966339    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-accelerators-collector-config\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.966523 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.966352    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.966523 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.966487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-wtmp\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.966905 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.966886    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-metrics-client-ca\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.969205 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.969156    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.969205 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.969163    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.969362 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.969300    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-tls\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.969635 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.969613    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.974787 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.974764    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7sln\" (UniqueName: \"kubernetes.io/projected/05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634-kube-api-access-g7sln\") pod \"kube-state-metrics-74cb57df7-8xxr2\" (UID: \"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.974963 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.974796    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkktd\" (UniqueName: \"kubernetes.io/projected/fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c-kube-api-access-qkktd\") pod \"node-exporter-fr5t8\" (UID: \"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c\") " pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:28.981567 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.981548    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"
Sep 08 14:00:28.990520 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:28.990499    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fr5t8"
Sep 08 14:00:29.002350 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:00:29.002316    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc89bfbe_e04e_4f5e_b2c6_ceda1b7a6d7c.slice/crio-5a82226f17a069efba380b3d930ad677eb03f52622ae5535f8deded637b93479 WatchSource:0}: Error finding container 5a82226f17a069efba380b3d930ad677eb03f52622ae5535f8deded637b93479: Status 404 returned error can't find the container with id 5a82226f17a069efba380b3d930ad677eb03f52622ae5535f8deded637b93479
Sep 08 14:00:29.270756 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:29.270725    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2"]
Sep 08 14:00:29.273991 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:00:29.273968    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05aecc9a_b6bc_4e7e_81ef_7ec2b4ae1634.slice/crio-6195aeee18be0e7f362a77efa7bed6b848ba23f0f8af95242f9cf241630ffe9f WatchSource:0}: Error finding container 6195aeee18be0e7f362a77efa7bed6b848ba23f0f8af95242f9cf241630ffe9f: Status 404 returned error can't find the container with id 6195aeee18be0e7f362a77efa7bed6b848ba23f0f8af95242f9cf241630ffe9f
Sep 08 14:00:29.368541 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:29.368513    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d9c7549-ca97-450c-9de8-9caf01336dbf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:29.370990 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:29.370968    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d9c7549-ca97-450c-9de8-9caf01336dbf-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-22mzn\" (UID: \"0d9c7549-ca97-450c-9de8-9caf01336dbf\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:29.443418 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:29.443382    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2" event={"ID":"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634","Type":"ContainerStarted","Data":"6195aeee18be0e7f362a77efa7bed6b848ba23f0f8af95242f9cf241630ffe9f"}
Sep 08 14:00:29.445436 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:29.445415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fr5t8" event={"ID":"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c","Type":"ContainerStarted","Data":"5a82226f17a069efba380b3d930ad677eb03f52622ae5535f8deded637b93479"}
Sep 08 14:00:29.537058 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:29.536975    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"
Sep 08 14:00:29.817251 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:29.817177    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn"]
Sep 08 14:00:29.820150 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:00:29.820122    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d9c7549_ca97_450c_9de8_9caf01336dbf.slice/crio-e528844688a19846d4cb6252ac0a9331d771db7fd31bab956b2ecb471fd0c0b0 WatchSource:0}: Error finding container e528844688a19846d4cb6252ac0a9331d771db7fd31bab956b2ecb471fd0c0b0: Status 404 returned error can't find the container with id e528844688a19846d4cb6252ac0a9331d771db7fd31bab956b2ecb471fd0c0b0
Sep 08 14:00:29.959355 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:29.959321    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab" path="/var/lib/kubelet/pods/ab3b46b5-1b4c-4393-8554-ebbc9f3a09ab/volumes"
Sep 08 14:00:30.451166 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:30.451125    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn" event={"ID":"0d9c7549-ca97-450c-9de8-9caf01336dbf","Type":"ContainerStarted","Data":"23497cc5ec8dbe81943435b672067ba9d4b61696b9a093a44cdf3c931c49d5b1"}
Sep 08 14:00:30.451558 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:30.451199    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn" event={"ID":"0d9c7549-ca97-450c-9de8-9caf01336dbf","Type":"ContainerStarted","Data":"d671e158837601774cb58aabe6f04de9e64f62e21f8a78434d8c08356c1533ef"}
Sep 08 14:00:30.451558 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:30.451227    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn" event={"ID":"0d9c7549-ca97-450c-9de8-9caf01336dbf","Type":"ContainerStarted","Data":"e528844688a19846d4cb6252ac0a9331d771db7fd31bab956b2ecb471fd0c0b0"}
Sep 08 14:00:31.456600 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:31.456565    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2" event={"ID":"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634","Type":"ContainerStarted","Data":"eee0f442b05a064cbb8caea93def77f04db896bb0be3b0164b5c246d7c48297f"}
Sep 08 14:00:31.456965 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:31.456630    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2" event={"ID":"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634","Type":"ContainerStarted","Data":"d69e2ccbb8a8713a32bbf50a56ecce8a76e478690ee34ac3bf08d6465216e021"}
Sep 08 14:00:31.456965 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:31.456652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2" event={"ID":"05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634","Type":"ContainerStarted","Data":"cd87a6d6fd16e9ab3403b375332b89eca6fc702cf53feba709b77d588fbf59e6"}
Sep 08 14:00:32.461382 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.461320    2564 generic.go:356] "Generic (PLEG): container finished" podID="fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c" containerID="1fd1e508ddfc1d5c400d36d28d3aed32e1704d54075b72256f54797ded5b1666" exitCode=0
Sep 08 14:00:32.463069 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.462142    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fr5t8" event={"ID":"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c","Type":"ContainerDied","Data":"1fd1e508ddfc1d5c400d36d28d3aed32e1704d54075b72256f54797ded5b1666"}
Sep 08 14:00:32.485540 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.485484    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-8xxr2" podStartSLOduration=3.192395348 podStartE2EDuration="4.485464503s" totalImagesPullingTime="1.293069155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:00:31.47801709 +0000 UTC m=+121.972987787" watchObservedRunningTime="2026-09-08 14:00:32.485464503 +0000 UTC m=+122.980435188"
Sep 08 14:00:32.956827 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.956798    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-78d9cb5d8-76zfh"]
Sep 08 14:00:32.959859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.959841    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:32.963334 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.963294    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 08 14:00:32.963459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.963372    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3qorud5olj47t\""
Sep 08 14:00:32.963459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.963382    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-9hn54\""
Sep 08 14:00:32.963459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.963404    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 08 14:00:32.963625 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.963485    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 08 14:00:32.963708 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.963694    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 08 14:00:32.973351 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.973331    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78d9cb5d8-76zfh"]
Sep 08 14:00:32.997898 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.997850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e702cd93-5a9c-4341-bcab-a74db83a2016-audit-log\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:32.997898 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.997907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e702cd93-5a9c-4341-bcab-a74db83a2016-client-ca-bundle\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:32.998099 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.997998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e702cd93-5a9c-4341-bcab-a74db83a2016-secret-metrics-server-client-certs\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:32.998099 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.998040    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e702cd93-5a9c-4341-bcab-a74db83a2016-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:32.998099 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.998068    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv5pj\" (UniqueName: \"kubernetes.io/projected/e702cd93-5a9c-4341-bcab-a74db83a2016-kube-api-access-lv5pj\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:32.998239 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.998109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e702cd93-5a9c-4341-bcab-a74db83a2016-secret-metrics-server-tls\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:32.998239 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:32.998136    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e702cd93-5a9c-4341-bcab-a74db83a2016-metrics-server-audit-profiles\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.099281 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.099246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e702cd93-5a9c-4341-bcab-a74db83a2016-secret-metrics-server-client-certs\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.099281 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.099285    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e702cd93-5a9c-4341-bcab-a74db83a2016-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.099506 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.099302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv5pj\" (UniqueName: \"kubernetes.io/projected/e702cd93-5a9c-4341-bcab-a74db83a2016-kube-api-access-lv5pj\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.099506 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.099362    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e702cd93-5a9c-4341-bcab-a74db83a2016-secret-metrics-server-tls\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.099506 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.099418    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e702cd93-5a9c-4341-bcab-a74db83a2016-metrics-server-audit-profiles\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.099506 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.099474    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e702cd93-5a9c-4341-bcab-a74db83a2016-audit-log\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.099506 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.099497    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e702cd93-5a9c-4341-bcab-a74db83a2016-client-ca-bundle\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.100074 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.100050    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e702cd93-5a9c-4341-bcab-a74db83a2016-audit-log\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.100351 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.100130    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e702cd93-5a9c-4341-bcab-a74db83a2016-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.100432 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.100352    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e702cd93-5a9c-4341-bcab-a74db83a2016-metrics-server-audit-profiles\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.102008 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.101989    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e702cd93-5a9c-4341-bcab-a74db83a2016-client-ca-bundle\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.102390 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.102376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e702cd93-5a9c-4341-bcab-a74db83a2016-secret-metrics-server-tls\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.102581 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.102562    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e702cd93-5a9c-4341-bcab-a74db83a2016-secret-metrics-server-client-certs\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.108123 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.108106    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv5pj\" (UniqueName: \"kubernetes.io/projected/e702cd93-5a9c-4341-bcab-a74db83a2016-kube-api-access-lv5pj\") pod \"metrics-server-78d9cb5d8-76zfh\" (UID: \"e702cd93-5a9c-4341-bcab-a74db83a2016\") " pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.269381 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.269347    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:33.467939 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.467893    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fr5t8" event={"ID":"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c","Type":"ContainerStarted","Data":"82d88b97b7c86b535d626fb8077af950226893be62d3b2765129aac09a9dbb1c"}
Sep 08 14:00:33.468372 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.467966    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fr5t8" event={"ID":"fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c","Type":"ContainerStarted","Data":"1557c7f1fe64b193426a77e4448d21dd373867748e661dce78647c094cf9e0db"}
Sep 08 14:00:33.470197 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.470168    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn" event={"ID":"0d9c7549-ca97-450c-9de8-9caf01336dbf","Type":"ContainerStarted","Data":"7a0bec2ce4c026b1d12d9785898fec4d8807914f3b91a2c4121de245f2301f60"}
Sep 08 14:00:33.497597 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.497548    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fr5t8" podStartSLOduration=2.952394922 podStartE2EDuration="5.497533457s" totalImagesPullingTime="2.472570702s" totalInitContainerRuntime="72.567833ms" isStatefulPod=true podCreationTimestamp="2026-09-08 14:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:00:33.496261291 +0000 UTC m=+123.991231976" watchObservedRunningTime="2026-09-08 14:00:33.497533457 +0000 UTC m=+123.992504161"
Sep 08 14:00:33.517167 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.517124    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-22mzn" podStartSLOduration=3.073905858 podStartE2EDuration="5.517108231s" totalImagesPullingTime="2.443202373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:00:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:00:33.51624824 +0000 UTC m=+124.011218924" watchObservedRunningTime="2026-09-08 14:00:33.517108231 +0000 UTC m=+124.012078915"
Sep 08 14:00:33.547906 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.547815    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78d9cb5d8-76zfh"]
Sep 08 14:00:33.550655 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:00:33.550630    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode702cd93_5a9c_4341_bcab_a74db83a2016.slice/crio-72d23e2155426c3aabde18c3046763c79246f87c69682a8f21c1f0207ff46620 WatchSource:0}: Error finding container 72d23e2155426c3aabde18c3046763c79246f87c69682a8f21c1f0207ff46620: Status 404 returned error can't find the container with id 72d23e2155426c3aabde18c3046763c79246f87c69682a8f21c1f0207ff46620
Sep 08 14:00:33.912144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.912046    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6456bdbb5d-m242q"]
Sep 08 14:00:33.916530 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.916505    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:33.919174 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.919153    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-txl89\""
Sep 08 14:00:33.919297 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.919205    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 08 14:00:33.919297 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.919283    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 08 14:00:33.919469 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.919454    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 08 14:00:33.919781 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.919768    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 08 14:00:33.919962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.919947    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 08 14:00:33.926000 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.925982    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 08 14:00:33.935051 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:33.935031    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6456bdbb5d-m242q"]
Sep 08 14:00:34.007019 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.006989    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90c0192f-f1a3-4417-b3e9-b97b41744f50-serving-certs-ca-bundle\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.007019 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.007024    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xz56\" (UniqueName: \"kubernetes.io/projected/90c0192f-f1a3-4417-b3e9-b97b41744f50-kube-api-access-5xz56\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.007210 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.007057    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90c0192f-f1a3-4417-b3e9-b97b41744f50-metrics-client-ca\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.007210 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.007076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.007210 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.007133    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90c0192f-f1a3-4417-b3e9-b97b41744f50-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.007210 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.007199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-telemeter-client-tls\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.007326 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.007218    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-secret-telemeter-client\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.007326 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.007245    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-federate-client-tls\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.108035 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.108002    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xz56\" (UniqueName: \"kubernetes.io/projected/90c0192f-f1a3-4417-b3e9-b97b41744f50-kube-api-access-5xz56\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.108188 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.108054    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90c0192f-f1a3-4417-b3e9-b97b41744f50-metrics-client-ca\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.108188 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.108087    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.108188 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.108127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90c0192f-f1a3-4417-b3e9-b97b41744f50-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.108188 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.108158    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-telemeter-client-tls\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.108188 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.108181    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-secret-telemeter-client\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.108438 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.108216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-federate-client-tls\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.108438 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.108262    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90c0192f-f1a3-4417-b3e9-b97b41744f50-serving-certs-ca-bundle\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.108928 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.108897    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90c0192f-f1a3-4417-b3e9-b97b41744f50-metrics-client-ca\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.109098 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.109027    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90c0192f-f1a3-4417-b3e9-b97b41744f50-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.109098 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.109037    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90c0192f-f1a3-4417-b3e9-b97b41744f50-serving-certs-ca-bundle\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.111392 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.111361    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.111492 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.111475    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-secret-telemeter-client\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.111561 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.111499    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-federate-client-tls\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.111561 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.111518    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/90c0192f-f1a3-4417-b3e9-b97b41744f50-telemeter-client-tls\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.117781 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.117756    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xz56\" (UniqueName: \"kubernetes.io/projected/90c0192f-f1a3-4417-b3e9-b97b41744f50-kube-api-access-5xz56\") pod \"telemeter-client-6456bdbb5d-m242q\" (UID: \"90c0192f-f1a3-4417-b3e9-b97b41744f50\") " pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.227780 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.227742    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q"
Sep 08 14:00:34.475668 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.475628    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh" event={"ID":"e702cd93-5a9c-4341-bcab-a74db83a2016","Type":"ContainerStarted","Data":"72d23e2155426c3aabde18c3046763c79246f87c69682a8f21c1f0207ff46620"}
Sep 08 14:00:34.521828 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.521699    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6456bdbb5d-m242q"]
Sep 08 14:00:34.526084 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:00:34.526059    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90c0192f_f1a3_4417_b3e9_b97b41744f50.slice/crio-2cb1b7ac779fe1ad4734901962a15ad64e46971101daead33e80d79c2ae7e720 WatchSource:0}: Error finding container 2cb1b7ac779fe1ad4734901962a15ad64e46971101daead33e80d79c2ae7e720: Status 404 returned error can't find the container with id 2cb1b7ac779fe1ad4734901962a15ad64e46971101daead33e80d79c2ae7e720
Sep 08 14:00:34.985342 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.985316    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:00:34.989875 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.989841    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:34.992908 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.992888    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 08 14:00:34.993072 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.992889    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 08 14:00:34.993559 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.993544    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 08 14:00:34.993776 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.993762    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 08 14:00:34.994038 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.994001    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 08 14:00:34.994120 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.994063    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 08 14:00:34.994171 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.994129    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 08 14:00:34.994171 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.994145    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 08 14:00:34.994171 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.994144    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 08 14:00:34.994314 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.994129    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 08 14:00:34.994314 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.994270    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 08 14:00:34.994314 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.994302    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2279qvqg8rg8j\""
Sep 08 14:00:34.994634 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.994618    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-48hzl\""
Sep 08 14:00:34.998630 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.998614    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 08 14:00:34.998674 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:34.998649    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 08 14:00:35.000576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.000561    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 08 14:00:35.016256 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016229    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016324 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016324 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016289    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016324 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-config\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016457 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016383    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016457 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016442    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016531 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016468    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016531 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016499    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-web-config\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016618 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016531    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016618 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016568    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016618 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016613    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-config-out\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016744 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016652    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016744 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016704    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016842 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016757    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016842 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016790    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.016842 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016813    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.017016 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.017016 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016912    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skc92\" (UniqueName: \"kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-kube-api-access-skc92\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.017016 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.017016 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.016971    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.093179 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.093141    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:00:35.118043 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118220 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118071    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118220 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118102    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118220 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118220 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118143    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118220 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skc92\" (UniqueName: \"kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-kube-api-access-skc92\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118220 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118209    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118327    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118357    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118395    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-config\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118422    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118444    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118468    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-web-config\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.118524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118523    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.119196 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.119196 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118592    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-config-out\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.119196 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.118620    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.119196 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.119142    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.119406 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.119252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.122296 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.119951    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.122296 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.120633    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.122296 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.121286    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.122296 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.121795    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.122296 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.122019    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.122296 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.122282    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.122677 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.122569    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.123305 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.123262    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.124140 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.124112    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.124586 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.124441    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-config\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.126004 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.125980    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.126366 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.126328    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-config-out\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.126767 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.126745    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-web-config\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.127951 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.127919    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.128198 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.128176    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.129193 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.129169    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skc92\" (UniqueName: \"kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-kube-api-access-skc92\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.129302 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.129224    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.129565 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.129538    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.308764 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.308730    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:35.482102 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.482060    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q" event={"ID":"90c0192f-f1a3-4417-b3e9-b97b41744f50","Type":"ContainerStarted","Data":"2cb1b7ac779fe1ad4734901962a15ad64e46971101daead33e80d79c2ae7e720"}
Sep 08 14:00:35.483810 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.483780    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh" event={"ID":"e702cd93-5a9c-4341-bcab-a74db83a2016","Type":"ContainerStarted","Data":"7dcbd5ac5f37200fe43b7ae269eb5e3a8d402e06b61db9e2b6ee6cf4f5bec336"}
Sep 08 14:00:35.505356 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.505306    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh" podStartSLOduration=2.110396157 podStartE2EDuration="3.505291264s" totalImagesPullingTime="1.394895107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:00:35.504295299 +0000 UTC m=+125.999265982" watchObservedRunningTime="2026-09-08 14:00:35.505291264 +0000 UTC m=+126.000261947"
Sep 08 14:00:35.599719 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:35.599640    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:00:35.602906 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:00:35.602855    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74026637_bd36_44ba_911c_778c40094950.slice/crio-abfe8c107e6ffe58b1d37e34028bdca15b78b1723cbdd255d19f3848692606ff WatchSource:0}: Error finding container abfe8c107e6ffe58b1d37e34028bdca15b78b1723cbdd255d19f3848692606ff: Status 404 returned error can't find the container with id abfe8c107e6ffe58b1d37e34028bdca15b78b1723cbdd255d19f3848692606ff
Sep 08 14:00:36.491130 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:36.491092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q" event={"ID":"90c0192f-f1a3-4417-b3e9-b97b41744f50","Type":"ContainerStarted","Data":"ab2600aee82a5850f4e8c38dc2f95fdd37f631704136f66cd15905d1fc4031ff"}
Sep 08 14:00:36.492278 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:36.492249    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerStarted","Data":"abfe8c107e6ffe58b1d37e34028bdca15b78b1723cbdd255d19f3848692606ff"}
Sep 08 14:00:38.502289 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:38.502197    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q" event={"ID":"90c0192f-f1a3-4417-b3e9-b97b41744f50","Type":"ContainerStarted","Data":"f6830628d6995bab611e1fe05eb22f93e78db4a7647aa77983452d22fa949c5b"}
Sep 08 14:00:38.502736 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:38.502326    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q" event={"ID":"90c0192f-f1a3-4417-b3e9-b97b41744f50","Type":"ContainerStarted","Data":"e050d8ecad45a0343cb30c3eb815d4e569ab73d819a2c5189abeb5b12027d1f2"}
Sep 08 14:00:38.503745 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:38.503717    2564 generic.go:356] "Generic (PLEG): container finished" podID="74026637-bd36-44ba-911c-778c40094950" containerID="0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37" exitCode=0
Sep 08 14:00:38.503841 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:38.503773    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerDied","Data":"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"}
Sep 08 14:00:38.527986 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:38.527936    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6456bdbb5d-m242q" podStartSLOduration=2.646865835 podStartE2EDuration="5.527918503s" totalImagesPullingTime="2.881052668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:00:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:00:38.526322719 +0000 UTC m=+129.021293404" watchObservedRunningTime="2026-09-08 14:00:38.527918503 +0000 UTC m=+129.022889187"
Sep 08 14:00:39.764891 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:39.764833    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 14:00:39.768050 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:39.768014    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa416b-05ee-4410-a741-c433bde86ee0-metrics-certs\") pod \"network-metrics-daemon-zkd7c\" (UID: \"a9fa416b-05ee-4410-a741-c433bde86ee0\") " pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 14:00:40.066194 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:40.066112    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-r4t74\""
Sep 08 14:00:40.074305 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:40.074273    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zkd7c"
Sep 08 14:00:40.396345 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:40.396266    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zkd7c"]
Sep 08 14:00:40.399556 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:00:40.399526    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9fa416b_05ee_4410_a741_c433bde86ee0.slice/crio-8cd862957742d513ed7a731ef0b3706f7d4be928eac6ce3ea6613729982b4e44 WatchSource:0}: Error finding container 8cd862957742d513ed7a731ef0b3706f7d4be928eac6ce3ea6613729982b4e44: Status 404 returned error can't find the container with id 8cd862957742d513ed7a731ef0b3706f7d4be928eac6ce3ea6613729982b4e44
Sep 08 14:00:40.517582 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:40.517543    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zkd7c" event={"ID":"a9fa416b-05ee-4410-a741-c433bde86ee0","Type":"ContainerStarted","Data":"8cd862957742d513ed7a731ef0b3706f7d4be928eac6ce3ea6613729982b4e44"}
Sep 08 14:00:42.528053 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:42.528010    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerStarted","Data":"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"}
Sep 08 14:00:42.528484 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:42.528083    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerStarted","Data":"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"}
Sep 08 14:00:42.529756 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:42.529734    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zkd7c" event={"ID":"a9fa416b-05ee-4410-a741-c433bde86ee0","Type":"ContainerStarted","Data":"abbf916977198f668ac6771b6506608fabdb7963fe3a9494e0079480a84e7d36"}
Sep 08 14:00:42.529837 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:42.529779    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zkd7c" event={"ID":"a9fa416b-05ee-4410-a741-c433bde86ee0","Type":"ContainerStarted","Data":"1994c104fe0462fd8afd69aaa2b394f7d38c3478637816b9aa3b372aa8e2fe6b"}
Sep 08 14:00:42.554855 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:42.554810    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zkd7c" podStartSLOduration=130.75071014 podStartE2EDuration="2m12.554797484s" totalImagesPullingTime="1.804087344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:00:42.552781365 +0000 UTC m=+133.047752049" watchObservedRunningTime="2026-09-08 14:00:42.554797484 +0000 UTC m=+133.049768168"
Sep 08 14:00:45.545482 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:45.545446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerStarted","Data":"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"}
Sep 08 14:00:45.546029 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:45.545522    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerStarted","Data":"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"}
Sep 08 14:00:45.546029 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:45.545583    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerStarted","Data":"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"}
Sep 08 14:00:45.546029 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:45.545627    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerStarted","Data":"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"}
Sep 08 14:00:45.583299 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:45.583245    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.327544251 podStartE2EDuration="11.583226996s" totalImagesPullingTime="8.230951239s" totalInitContainerRuntime="24.731506ms" isStatefulPod=false podCreationTimestamp="2026-09-08 14:00:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:00:45.579752429 +0000 UTC m=+136.074723118" watchObservedRunningTime="2026-09-08 14:00:45.583226996 +0000 UTC m=+136.078197683"
Sep 08 14:00:50.309365 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:50.309331    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:00:53.269644 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:53.269606    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:53.270059 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:53.269683    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:00:55.547686 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:00:55.547656    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qsrr6_d4066c24-5beb-4011-a94b-cf594601f554/serve-healthcheck-canary/0.log"
Sep 08 14:01:13.275795 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:13.275765    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:01:13.279840 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:13.279812    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-78d9cb5d8-76zfh"
Sep 08 14:01:35.309725 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:35.309640    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:35.329818 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:35.329795    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:35.757246 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:35.757220    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.348212 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.348180    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:01:53.349184 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.349144    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy-thanos" containerID="cri-o://67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6" gracePeriod=600
Sep 08 14:01:53.349466 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.349149    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="prometheus" containerID="cri-o://7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be" gracePeriod=600
Sep 08 14:01:53.349662 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.349190    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="thanos-sidecar" containerID="cri-o://abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b" gracePeriod=600
Sep 08 14:01:53.349662 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.349207    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="config-reloader" containerID="cri-o://92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef" gracePeriod=600
Sep 08 14:01:53.349662 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.349236    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy" containerID="cri-o://346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f" gracePeriod=600
Sep 08 14:01:53.349662 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.349244    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy-web" containerID="cri-o://edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445" gracePeriod=600
Sep 08 14:01:53.590241 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.590218    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.677113 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677035    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-config-out\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677113 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677071    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-0\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677113 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677089    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-kube-rbac-proxy\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677115    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677145    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-config\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677174    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-db\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677199    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-metrics-client-ca\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677227    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-metrics-client-certs\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677261    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-kubelet-serving-ca-bundle\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677285    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-2\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677310    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skc92\" (UniqueName: \"kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-kube-api-access-skc92\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677343    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677373    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-1\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677894 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677402    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-trusted-ca-bundle\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677894 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677443    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-serving-certs-ca-bundle\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677894 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677482    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-grpc-tls\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677894 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677513    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-tls\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677894 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677543    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-web-config\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677894 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677625    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-tls-assets\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.677894 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677654    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-thanos-prometheus-http-client-file\") pod \"74026637-bd36-44ba-911c-778c40094950\" (UID: \"74026637-bd36-44ba-911c-778c40094950\") "
Sep 08 14:01:53.678224 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.677979    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:01:53.678341 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.678295    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:01:53.678797 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.678563    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:01:53.678797 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.678620    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:01:53.679215 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.678996    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:01:53.679469 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.679313    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:01:53.679469 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.679319    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:01:53.679469 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.679412    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:01:53.681302 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.681265    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:01:53.681452 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.681426    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:01:53.682427 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.682399    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-config-out" (OuterVolumeSpecName: "config-out") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:01:53.682427 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.682413    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-config" (OuterVolumeSpecName: "config") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:01:53.682743 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.682469    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:01:53.683544 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.683519    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:01:53.683544 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.683516    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-kube-api-access-skc92" (OuterVolumeSpecName: "kube-api-access-skc92") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "kube-api-access-skc92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:01:53.683707 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.683626    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:01:53.683707 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.683660    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:01:53.684636 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.684602    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:01:53.684703 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.684632    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:01:53.693098 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.693080    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-web-config" (OuterVolumeSpecName: "web-config") pod "74026637-bd36-44ba-911c-778c40094950" (UID: "74026637-bd36-44ba-911c-778c40094950"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:01:53.778779 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778751    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778779 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778779    2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-grpc-tls\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778788    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778800    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-web-config\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778810    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-tls-assets\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778818    2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778825    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-config-out\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778833    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778840    2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-kube-rbac-proxy\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778850    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778858    2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-config\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778886    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-db\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778893    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-metrics-client-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778902    2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-metrics-client-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778910    2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778919    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778927    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skc92\" (UniqueName: \"kubernetes.io/projected/74026637-bd36-44ba-911c-778c40094950-kube-api-access-skc92\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778936    2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/74026637-bd36-44ba-911c-778c40094950-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778944    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.778962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.778952    2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74026637-bd36-44ba-911c-778c40094950-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:01:53.814247 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814218    2564 generic.go:356] "Generic (PLEG): container finished" podID="74026637-bd36-44ba-911c-778c40094950" containerID="67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6" exitCode=0
Sep 08 14:01:53.814247 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814244    2564 generic.go:356] "Generic (PLEG): container finished" podID="74026637-bd36-44ba-911c-778c40094950" containerID="346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f" exitCode=0
Sep 08 14:01:53.814247 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814251    2564 generic.go:356] "Generic (PLEG): container finished" podID="74026637-bd36-44ba-911c-778c40094950" containerID="edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445" exitCode=0
Sep 08 14:01:53.814428 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814257    2564 generic.go:356] "Generic (PLEG): container finished" podID="74026637-bd36-44ba-911c-778c40094950" containerID="abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b" exitCode=0
Sep 08 14:01:53.814428 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814262    2564 generic.go:356] "Generic (PLEG): container finished" podID="74026637-bd36-44ba-911c-778c40094950" containerID="92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef" exitCode=0
Sep 08 14:01:53.814428 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814267    2564 generic.go:356] "Generic (PLEG): container finished" podID="74026637-bd36-44ba-911c-778c40094950" containerID="7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be" exitCode=0
Sep 08 14:01:53.814428 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814290    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.814428 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814300    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerDied","Data":"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"}
Sep 08 14:01:53.814428 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814375    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerDied","Data":"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"}
Sep 08 14:01:53.814428 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814409    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerDied","Data":"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"}
Sep 08 14:01:53.814626 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerDied","Data":"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"}
Sep 08 14:01:53.814626 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814452    2564 scope.go:117] "RemoveContainer" containerID="67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"
Sep 08 14:01:53.814626 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814485    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerDied","Data":"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"}
Sep 08 14:01:53.814626 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814524    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerDied","Data":"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"}
Sep 08 14:01:53.814626 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.814566    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"74026637-bd36-44ba-911c-778c40094950","Type":"ContainerDied","Data":"abfe8c107e6ffe58b1d37e34028bdca15b78b1723cbdd255d19f3848692606ff"}
Sep 08 14:01:53.823089 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.823069    2564 scope.go:117] "RemoveContainer" containerID="346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"
Sep 08 14:01:53.830006 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.829985    2564 scope.go:117] "RemoveContainer" containerID="edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"
Sep 08 14:01:53.836616 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.836599    2564 scope.go:117] "RemoveContainer" containerID="abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"
Sep 08 14:01:53.841851 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.841831    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:01:53.844297 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.844259    2564 scope.go:117] "RemoveContainer" containerID="92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"
Sep 08 14:01:53.844950 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.844927    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:01:53.851076 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.851048    2564 scope.go:117] "RemoveContainer" containerID="7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"
Sep 08 14:01:53.858316 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.858301    2564 scope.go:117] "RemoveContainer" containerID="0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"
Sep 08 14:01:53.865181 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.865167    2564 scope.go:117] "RemoveContainer" containerID="67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"
Sep 08 14:01:53.865448 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:01:53.865430    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": container with ID starting with 67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6 not found: ID does not exist" containerID="67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"
Sep 08 14:01:53.865510 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.865456    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"} err="failed to get container status \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": rpc error: code = NotFound desc = could not find container \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": container with ID starting with 67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6 not found: ID does not exist"
Sep 08 14:01:53.865510 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.865476    2564 scope.go:117] "RemoveContainer" containerID="346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"
Sep 08 14:01:53.865739 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:01:53.865724    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": container with ID starting with 346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f not found: ID does not exist" containerID="346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"
Sep 08 14:01:53.865777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.865745    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"} err="failed to get container status \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": rpc error: code = NotFound desc = could not find container \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": container with ID starting with 346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f not found: ID does not exist"
Sep 08 14:01:53.865777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.865764    2564 scope.go:117] "RemoveContainer" containerID="edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"
Sep 08 14:01:53.866040 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:01:53.866025    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": container with ID starting with edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445 not found: ID does not exist" containerID="edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"
Sep 08 14:01:53.866079 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.866044    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"} err="failed to get container status \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": rpc error: code = NotFound desc = could not find container \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": container with ID starting with edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445 not found: ID does not exist"
Sep 08 14:01:53.866079 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.866059    2564 scope.go:117] "RemoveContainer" containerID="abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"
Sep 08 14:01:53.866296 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:01:53.866282    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": container with ID starting with abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b not found: ID does not exist" containerID="abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"
Sep 08 14:01:53.866342 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.866297    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"} err="failed to get container status \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": rpc error: code = NotFound desc = could not find container \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": container with ID starting with abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b not found: ID does not exist"
Sep 08 14:01:53.866342 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.866309    2564 scope.go:117] "RemoveContainer" containerID="92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"
Sep 08 14:01:53.866502 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:01:53.866485    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": container with ID starting with 92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef not found: ID does not exist" containerID="92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"
Sep 08 14:01:53.866578 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.866505    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"} err="failed to get container status \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": rpc error: code = NotFound desc = could not find container \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": container with ID starting with 92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef not found: ID does not exist"
Sep 08 14:01:53.866578 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.866517    2564 scope.go:117] "RemoveContainer" containerID="7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"
Sep 08 14:01:53.866712 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:01:53.866697    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": container with ID starting with 7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be not found: ID does not exist" containerID="7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"
Sep 08 14:01:53.866748 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.866716    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"} err="failed to get container status \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": rpc error: code = NotFound desc = could not find container \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": container with ID starting with 7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be not found: ID does not exist"
Sep 08 14:01:53.866748 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.866727    2564 scope.go:117] "RemoveContainer" containerID="0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"
Sep 08 14:01:53.866990 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:01:53.866974    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": container with ID starting with 0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37 not found: ID does not exist" containerID="0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"
Sep 08 14:01:53.867045 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.866992    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"} err="failed to get container status \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": rpc error: code = NotFound desc = could not find container \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": container with ID starting with 0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37 not found: ID does not exist"
Sep 08 14:01:53.867045 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.867009    2564 scope.go:117] "RemoveContainer" containerID="67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"
Sep 08 14:01:53.867245 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.867230    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"} err="failed to get container status \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": rpc error: code = NotFound desc = could not find container \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": container with ID starting with 67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6 not found: ID does not exist"
Sep 08 14:01:53.867286 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.867245    2564 scope.go:117] "RemoveContainer" containerID="346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"
Sep 08 14:01:53.867451 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.867436    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"} err="failed to get container status \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": rpc error: code = NotFound desc = could not find container \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": container with ID starting with 346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f not found: ID does not exist"
Sep 08 14:01:53.867494 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.867452    2564 scope.go:117] "RemoveContainer" containerID="edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"
Sep 08 14:01:53.867688 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.867672    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"} err="failed to get container status \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": rpc error: code = NotFound desc = could not find container \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": container with ID starting with edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445 not found: ID does not exist"
Sep 08 14:01:53.867728 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.867690    2564 scope.go:117] "RemoveContainer" containerID="abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"
Sep 08 14:01:53.867924 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.867909    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"} err="failed to get container status \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": rpc error: code = NotFound desc = could not find container \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": container with ID starting with abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b not found: ID does not exist"
Sep 08 14:01:53.867966 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.867924    2564 scope.go:117] "RemoveContainer" containerID="92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"
Sep 08 14:01:53.868108 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.868090    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"} err="failed to get container status \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": rpc error: code = NotFound desc = could not find container \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": container with ID starting with 92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef not found: ID does not exist"
Sep 08 14:01:53.868141 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.868110    2564 scope.go:117] "RemoveContainer" containerID="7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"
Sep 08 14:01:53.868305 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.868291    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"} err="failed to get container status \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": rpc error: code = NotFound desc = could not find container \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": container with ID starting with 7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be not found: ID does not exist"
Sep 08 14:01:53.868341 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.868306    2564 scope.go:117] "RemoveContainer" containerID="0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"
Sep 08 14:01:53.868519 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.868504    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"} err="failed to get container status \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": rpc error: code = NotFound desc = could not find container \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": container with ID starting with 0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37 not found: ID does not exist"
Sep 08 14:01:53.868562 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.868519    2564 scope.go:117] "RemoveContainer" containerID="67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"
Sep 08 14:01:53.868697 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.868682    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"} err="failed to get container status \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": rpc error: code = NotFound desc = could not find container \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": container with ID starting with 67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6 not found: ID does not exist"
Sep 08 14:01:53.868738 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.868696    2564 scope.go:117] "RemoveContainer" containerID="346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"
Sep 08 14:01:53.868888 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.868849    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"} err="failed to get container status \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": rpc error: code = NotFound desc = could not find container \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": container with ID starting with 346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f not found: ID does not exist"
Sep 08 14:01:53.868935 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.868889    2564 scope.go:117] "RemoveContainer" containerID="edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"
Sep 08 14:01:53.869100 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.869085    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"} err="failed to get container status \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": rpc error: code = NotFound desc = could not find container \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": container with ID starting with edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445 not found: ID does not exist"
Sep 08 14:01:53.869158 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.869100    2564 scope.go:117] "RemoveContainer" containerID="abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"
Sep 08 14:01:53.869326 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.869312    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"} err="failed to get container status \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": rpc error: code = NotFound desc = could not find container \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": container with ID starting with abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b not found: ID does not exist"
Sep 08 14:01:53.869326 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.869326    2564 scope.go:117] "RemoveContainer" containerID="92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"
Sep 08 14:01:53.869528 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.869513    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"} err="failed to get container status \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": rpc error: code = NotFound desc = could not find container \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": container with ID starting with 92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef not found: ID does not exist"
Sep 08 14:01:53.869575 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.869528    2564 scope.go:117] "RemoveContainer" containerID="7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"
Sep 08 14:01:53.869729 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.869716    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"} err="failed to get container status \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": rpc error: code = NotFound desc = could not find container \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": container with ID starting with 7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be not found: ID does not exist"
Sep 08 14:01:53.869729 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.869728    2564 scope.go:117] "RemoveContainer" containerID="0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"
Sep 08 14:01:53.869968 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.869952    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"} err="failed to get container status \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": rpc error: code = NotFound desc = could not find container \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": container with ID starting with 0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37 not found: ID does not exist"
Sep 08 14:01:53.870015 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.869972    2564 scope.go:117] "RemoveContainer" containerID="67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"
Sep 08 14:01:53.870216 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.870200    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"} err="failed to get container status \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": rpc error: code = NotFound desc = could not find container \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": container with ID starting with 67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6 not found: ID does not exist"
Sep 08 14:01:53.870252 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.870225    2564 scope.go:117] "RemoveContainer" containerID="346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"
Sep 08 14:01:53.870436 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.870415    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"} err="failed to get container status \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": rpc error: code = NotFound desc = could not find container \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": container with ID starting with 346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f not found: ID does not exist"
Sep 08 14:01:53.870476 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.870437    2564 scope.go:117] "RemoveContainer" containerID="edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"
Sep 08 14:01:53.870663 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.870646    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"} err="failed to get container status \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": rpc error: code = NotFound desc = could not find container \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": container with ID starting with edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445 not found: ID does not exist"
Sep 08 14:01:53.870699 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.870665    2564 scope.go:117] "RemoveContainer" containerID="abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"
Sep 08 14:01:53.870891 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.870855    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"} err="failed to get container status \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": rpc error: code = NotFound desc = could not find container \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": container with ID starting with abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b not found: ID does not exist"
Sep 08 14:01:53.870927 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.870885    2564 scope.go:117] "RemoveContainer" containerID="92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"
Sep 08 14:01:53.871108 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.871091    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"} err="failed to get container status \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": rpc error: code = NotFound desc = could not find container \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": container with ID starting with 92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef not found: ID does not exist"
Sep 08 14:01:53.871144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.871108    2564 scope.go:117] "RemoveContainer" containerID="7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"
Sep 08 14:01:53.871324 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.871308    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"} err="failed to get container status \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": rpc error: code = NotFound desc = could not find container \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": container with ID starting with 7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be not found: ID does not exist"
Sep 08 14:01:53.871359 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.871325    2564 scope.go:117] "RemoveContainer" containerID="0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"
Sep 08 14:01:53.871522 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.871508    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"} err="failed to get container status \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": rpc error: code = NotFound desc = could not find container \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": container with ID starting with 0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37 not found: ID does not exist"
Sep 08 14:01:53.871554 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.871522    2564 scope.go:117] "RemoveContainer" containerID="67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"
Sep 08 14:01:53.871742 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.871729    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"} err="failed to get container status \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": rpc error: code = NotFound desc = could not find container \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": container with ID starting with 67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6 not found: ID does not exist"
Sep 08 14:01:53.871778 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.871743    2564 scope.go:117] "RemoveContainer" containerID="346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"
Sep 08 14:01:53.872001 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.871986    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"} err="failed to get container status \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": rpc error: code = NotFound desc = could not find container \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": container with ID starting with 346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f not found: ID does not exist"
Sep 08 14:01:53.872053 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.872001    2564 scope.go:117] "RemoveContainer" containerID="edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"
Sep 08 14:01:53.872222 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.872203    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"} err="failed to get container status \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": rpc error: code = NotFound desc = could not find container \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": container with ID starting with edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445 not found: ID does not exist"
Sep 08 14:01:53.872268 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.872224    2564 scope.go:117] "RemoveContainer" containerID="abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"
Sep 08 14:01:53.872463 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.872447    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"} err="failed to get container status \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": rpc error: code = NotFound desc = could not find container \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": container with ID starting with abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b not found: ID does not exist"
Sep 08 14:01:53.872504 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.872463    2564 scope.go:117] "RemoveContainer" containerID="92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"
Sep 08 14:01:53.872684 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.872671    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"} err="failed to get container status \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": rpc error: code = NotFound desc = could not find container \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": container with ID starting with 92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef not found: ID does not exist"
Sep 08 14:01:53.872684 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.872684    2564 scope.go:117] "RemoveContainer" containerID="7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"
Sep 08 14:01:53.872920 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.872904    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"} err="failed to get container status \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": rpc error: code = NotFound desc = could not find container \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": container with ID starting with 7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be not found: ID does not exist"
Sep 08 14:01:53.872985 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.872920    2564 scope.go:117] "RemoveContainer" containerID="0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"
Sep 08 14:01:53.873166 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.873148    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"} err="failed to get container status \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": rpc error: code = NotFound desc = could not find container \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": container with ID starting with 0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37 not found: ID does not exist"
Sep 08 14:01:53.873166 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.873165    2564 scope.go:117] "RemoveContainer" containerID="67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"
Sep 08 14:01:53.873369 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.873353    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6"} err="failed to get container status \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": rpc error: code = NotFound desc = could not find container \"67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6\": container with ID starting with 67cef23835cab5cad12d79bc6c1b613ca9fda948f3a5f859382c53e8053db6c6 not found: ID does not exist"
Sep 08 14:01:53.873412 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.873369    2564 scope.go:117] "RemoveContainer" containerID="346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"
Sep 08 14:01:53.873590 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.873570    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f"} err="failed to get container status \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": rpc error: code = NotFound desc = could not find container \"346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f\": container with ID starting with 346d5b0397ce49473287bc99ca631d450669f2344171c5e89a80c79f2aa60a1f not found: ID does not exist"
Sep 08 14:01:53.873590 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.873587    2564 scope.go:117] "RemoveContainer" containerID="edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"
Sep 08 14:01:53.873845 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.873824    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445"} err="failed to get container status \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": rpc error: code = NotFound desc = could not find container \"edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445\": container with ID starting with edcbaab5e56733ccaca1d666195d60fc8dde3d23fddef315aa7ca5bf5998d445 not found: ID does not exist"
Sep 08 14:01:53.873922 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.873847    2564 scope.go:117] "RemoveContainer" containerID="abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"
Sep 08 14:01:53.874117 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.874101    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b"} err="failed to get container status \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": rpc error: code = NotFound desc = could not find container \"abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b\": container with ID starting with abf548f860906be4651c37ca1a78a6bd7ef84118124fd1995941e0961542a72b not found: ID does not exist"
Sep 08 14:01:53.874117 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.874116    2564 scope.go:117] "RemoveContainer" containerID="92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"
Sep 08 14:01:53.874349 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.874329    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef"} err="failed to get container status \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": rpc error: code = NotFound desc = could not find container \"92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef\": container with ID starting with 92a48e52062e972ec74ecb73af337ec97a980a7e8281a373bedeabb397640eef not found: ID does not exist"
Sep 08 14:01:53.874419 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.874352    2564 scope.go:117] "RemoveContainer" containerID="7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"
Sep 08 14:01:53.874620 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.874600    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be"} err="failed to get container status \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": rpc error: code = NotFound desc = could not find container \"7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be\": container with ID starting with 7a85e0602029f233d5d360df0e7b62a0540b56e73e8e2edab4efa35d4eb036be not found: ID does not exist"
Sep 08 14:01:53.874674 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.874620    2564 scope.go:117] "RemoveContainer" containerID="0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"
Sep 08 14:01:53.874893 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.874856    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37"} err="failed to get container status \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": rpc error: code = NotFound desc = could not find container \"0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37\": container with ID starting with 0e877a2cae53bf52f1aa89d6fe159ae5b709b49dc7888ffc86dd694345407c37 not found: ID does not exist"
Sep 08 14:01:53.880589 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.880564    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:01:53.881280 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881261    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy-web"
Sep 08 14:01:53.881364 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881282    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy-web"
Sep 08 14:01:53.881364 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881296    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy"
Sep 08 14:01:53.881364 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881304    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy"
Sep 08 14:01:53.881364 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881323    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="init-config-reloader"
Sep 08 14:01:53.881364 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881331    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="init-config-reloader"
Sep 08 14:01:53.881364 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881360    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="thanos-sidecar"
Sep 08 14:01:53.881632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881368    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="thanos-sidecar"
Sep 08 14:01:53.881632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881417    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="config-reloader"
Sep 08 14:01:53.881632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881425    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="config-reloader"
Sep 08 14:01:53.881632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881438    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="prometheus"
Sep 08 14:01:53.881632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881446    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="prometheus"
Sep 08 14:01:53.881632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881465    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy-thanos"
Sep 08 14:01:53.881632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881474    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy-thanos"
Sep 08 14:01:53.881950 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881717    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy-thanos"
Sep 08 14:01:53.881950 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881729    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="thanos-sidecar"
Sep 08 14:01:53.881950 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881737    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy"
Sep 08 14:01:53.881950 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881751    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="prometheus"
Sep 08 14:01:53.881950 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881797    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="config-reloader"
Sep 08 14:01:53.881950 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.881807    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="74026637-bd36-44ba-911c-778c40094950" containerName="kube-rbac-proxy-web"
Sep 08 14:01:53.887056 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.887036    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.894272 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.894251    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 08 14:01:53.894412 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.894387    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 08 14:01:53.894581 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.894561    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 08 14:01:53.904101 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.904079    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 08 14:01:53.904388 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.904369    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 08 14:01:53.904562 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.904492    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 08 14:01:53.904648 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.904594    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 08 14:01:53.904648 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.904614    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-48hzl\""
Sep 08 14:01:53.905485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.905468    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 08 14:01:53.905574 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.905468    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 08 14:01:53.905574 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.905471    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 08 14:01:53.910120 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.910102    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 08 14:01:53.910259 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.910246    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 08 14:01:53.928816 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.928758    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 08 14:01:53.928816 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.928798    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2279qvqg8rg8j\""
Sep 08 14:01:53.930622 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.930606    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:01:53.934257 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.934240    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 08 14:01:53.956996 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.956974    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74026637-bd36-44ba-911c-778c40094950" path="/var/lib/kubelet/pods/74026637-bd36-44ba-911c-778c40094950/volumes"
Sep 08 14:01:53.981232 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981206    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981343 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981236    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981343 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981259    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981343 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981280    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6whv\" (UniqueName: \"kubernetes.io/projected/82b894e0-d6ef-485f-8e7d-20dc049ac702-kube-api-access-f6whv\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981381    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981415    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981472    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981491    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981506    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-config\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981526    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981543    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981568    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-web-config\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981610    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981856 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981637    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82b894e0-d6ef-485f-8e7d-20dc049ac702-config-out\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981856 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981658    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981856 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981685    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:53.981856 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:53.981719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82b894e0-d6ef-485f-8e7d-20dc049ac702-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.082958 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.082909    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82b894e0-d6ef-485f-8e7d-20dc049ac702-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.082958 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.082957    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.083189 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.082988    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.083189 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.083025    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.083189 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.083054    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6whv\" (UniqueName: \"kubernetes.io/projected/82b894e0-d6ef-485f-8e7d-20dc049ac702-kube-api-access-f6whv\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.083330 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.083298    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.083428 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.083363    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.083428 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.083398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.083428 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.083423    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.083929 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.083835    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084028 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.083987    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084090 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084023    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084090 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084090 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084034    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084248 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084248 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-config\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084248 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084167    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084248 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084192    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084248 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084219    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-web-config\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084250    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82b894e0-d6ef-485f-8e7d-20dc049ac702-config-out\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084335    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084367    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084847 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.084950 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.084887    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.086078 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.085772    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.086328 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.086305    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82b894e0-d6ef-485f-8e7d-20dc049ac702-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.086608 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.086585    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.086810 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.086789    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.087431 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.087208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.087431 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.087369    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82b894e0-d6ef-485f-8e7d-20dc049ac702-config-out\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.087591 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.087536    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.087637 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.087598    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82b894e0-d6ef-485f-8e7d-20dc049ac702-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.089164 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.089140    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-web-config\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.089463 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.089440    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.089636 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.089616    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.089702 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.089620    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.089852 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.089833    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/82b894e0-d6ef-485f-8e7d-20dc049ac702-config\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.091120 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.091103    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6whv\" (UniqueName: \"kubernetes.io/projected/82b894e0-d6ef-485f-8e7d-20dc049ac702-kube-api-access-f6whv\") pod \"prometheus-k8s-0\" (UID: \"82b894e0-d6ef-485f-8e7d-20dc049ac702\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.196573 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.196494    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:01:54.475847 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.475814    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 14:01:54.480349 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:01:54.480322    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b894e0_d6ef_485f_8e7d_20dc049ac702.slice/crio-87d4ebe92916a87c26777872b1ab445d9ef5e0b3e9abf11b6fa9ac1689b703a2 WatchSource:0}: Error finding container 87d4ebe92916a87c26777872b1ab445d9ef5e0b3e9abf11b6fa9ac1689b703a2: Status 404 returned error can't find the container with id 87d4ebe92916a87c26777872b1ab445d9ef5e0b3e9abf11b6fa9ac1689b703a2
Sep 08 14:01:54.820557 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.820466    2564 generic.go:356] "Generic (PLEG): container finished" podID="82b894e0-d6ef-485f-8e7d-20dc049ac702" containerID="cd2a877a2939ae48936db30e9637dd66d0cb393c8e2ddce58697219ca2a701b9" exitCode=0
Sep 08 14:01:54.820557 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.820537    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82b894e0-d6ef-485f-8e7d-20dc049ac702","Type":"ContainerDied","Data":"cd2a877a2939ae48936db30e9637dd66d0cb393c8e2ddce58697219ca2a701b9"}
Sep 08 14:01:54.820790 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:54.820606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82b894e0-d6ef-485f-8e7d-20dc049ac702","Type":"ContainerStarted","Data":"87d4ebe92916a87c26777872b1ab445d9ef5e0b3e9abf11b6fa9ac1689b703a2"}
Sep 08 14:01:55.834755 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:55.834724    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82b894e0-d6ef-485f-8e7d-20dc049ac702","Type":"ContainerStarted","Data":"a4cdc781f2a65d0cc4ecc4668ef3180a56bbc47ef6e37dd995d8fb2c25218f42"}
Sep 08 14:01:55.835259 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:55.834807    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82b894e0-d6ef-485f-8e7d-20dc049ac702","Type":"ContainerStarted","Data":"42627cfb2b58dd4e8530fcf4883d18da7848a2cf58c09bcb1d95e289f6303d25"}
Sep 08 14:01:55.835259 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:55.834834    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82b894e0-d6ef-485f-8e7d-20dc049ac702","Type":"ContainerStarted","Data":"e76b92818017d8266ba2e15b5b1254b6d7d4e8c62c85b220d6163b650bf7b277"}
Sep 08 14:01:55.835259 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:55.834880    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82b894e0-d6ef-485f-8e7d-20dc049ac702","Type":"ContainerStarted","Data":"0afa54717d6da16878f4395e2f200b83eb3e6d5f033d773c81f61b1b01d1fa67"}
Sep 08 14:01:55.835259 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:55.834909    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82b894e0-d6ef-485f-8e7d-20dc049ac702","Type":"ContainerStarted","Data":"0fb976032a1f8b490cdb6de061128380761ab50c8b44240e03ce2c7f81b81c81"}
Sep 08 14:01:55.835259 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:55.834931    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82b894e0-d6ef-485f-8e7d-20dc049ac702","Type":"ContainerStarted","Data":"4c222a56dce70c2888cfac82b43938b460078bdfaef57b66c096405c2e8d828b"}
Sep 08 14:01:55.865191 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:55.865141    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.846949709 podStartE2EDuration="2.865124581s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.174872ms" isStatefulPod=false podCreationTimestamp="2026-09-08 14:01:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:01:55.862372031 +0000 UTC m=+206.357342719" watchObservedRunningTime="2026-09-08 14:01:55.865124581 +0000 UTC m=+206.360095265"
Sep 08 14:01:59.196827 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:01:59.196788    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:02:54.197636 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:02:54.197547    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:02:54.213920 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:02:54.213895    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:02:55.074598 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:02:55.074574    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 14:03:16.912801 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:16.912768    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-s9knl"]
Sep 08 14:03:16.914916 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:16.914899    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl"
Sep 08 14:03:16.918824 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:16.918787    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 08 14:03:16.918972 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:16.918949    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-l5bzp\""
Sep 08 14:03:16.919936 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:16.919922    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 08 14:03:16.924358 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:16.924337    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-s9knl"]
Sep 08 14:03:16.951027 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:16.950997    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7php\" (UniqueName: \"kubernetes.io/projected/8887f740-c623-48b9-9412-44bbdcbb6191-kube-api-access-n7php\") pod \"cert-manager-webhook-6486ffd997-s9knl\" (UID: \"8887f740-c623-48b9-9412-44bbdcbb6191\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl"
Sep 08 14:03:16.951157 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:16.951043    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8887f740-c623-48b9-9412-44bbdcbb6191-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-s9knl\" (UID: \"8887f740-c623-48b9-9412-44bbdcbb6191\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl"
Sep 08 14:03:17.051568 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.051530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7php\" (UniqueName: \"kubernetes.io/projected/8887f740-c623-48b9-9412-44bbdcbb6191-kube-api-access-n7php\") pod \"cert-manager-webhook-6486ffd997-s9knl\" (UID: \"8887f740-c623-48b9-9412-44bbdcbb6191\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl"
Sep 08 14:03:17.051742 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.051585    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8887f740-c623-48b9-9412-44bbdcbb6191-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-s9knl\" (UID: \"8887f740-c623-48b9-9412-44bbdcbb6191\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl"
Sep 08 14:03:17.065509 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.065488    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8887f740-c623-48b9-9412-44bbdcbb6191-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-s9knl\" (UID: \"8887f740-c623-48b9-9412-44bbdcbb6191\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl"
Sep 08 14:03:17.067090 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.067070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7php\" (UniqueName: \"kubernetes.io/projected/8887f740-c623-48b9-9412-44bbdcbb6191-kube-api-access-n7php\") pod \"cert-manager-webhook-6486ffd997-s9knl\" (UID: \"8887f740-c623-48b9-9412-44bbdcbb6191\") " pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl"
Sep 08 14:03:17.224203 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.224171    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl"
Sep 08 14:03:17.522058 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.521983    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-s9knl"]
Sep 08 14:03:17.534761 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:03:17.534726    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8887f740_c623_48b9_9412_44bbdcbb6191.slice/crio-a57d42a5325c0daebbb696c7aa55acb182c4a8b17739b13eefa31ba77b77e73e WatchSource:0}: Error finding container a57d42a5325c0daebbb696c7aa55acb182c4a8b17739b13eefa31ba77b77e73e: Status 404 returned error can't find the container with id a57d42a5325c0daebbb696c7aa55acb182c4a8b17739b13eefa31ba77b77e73e
Sep 08 14:03:17.632605 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.632574    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5"]
Sep 08 14:03:17.635327 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.635310    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5"
Sep 08 14:03:17.638513 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.638493    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8xmfn\""
Sep 08 14:03:17.656102 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.656077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8310741a-cff9-4f62-969a-eda2b171a1e6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-x9sk5\" (UID: \"8310741a-cff9-4f62-969a-eda2b171a1e6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5"
Sep 08 14:03:17.656208 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.656120    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhhqd\" (UniqueName: \"kubernetes.io/projected/8310741a-cff9-4f62-969a-eda2b171a1e6-kube-api-access-jhhqd\") pod \"cert-manager-cainjector-559cdd8c87-x9sk5\" (UID: \"8310741a-cff9-4f62-969a-eda2b171a1e6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5"
Sep 08 14:03:17.703054 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.703027    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5"]
Sep 08 14:03:17.757075 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.757047    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8310741a-cff9-4f62-969a-eda2b171a1e6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-x9sk5\" (UID: \"8310741a-cff9-4f62-969a-eda2b171a1e6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5"
Sep 08 14:03:17.757208 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.757092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhhqd\" (UniqueName: \"kubernetes.io/projected/8310741a-cff9-4f62-969a-eda2b171a1e6-kube-api-access-jhhqd\") pod \"cert-manager-cainjector-559cdd8c87-x9sk5\" (UID: \"8310741a-cff9-4f62-969a-eda2b171a1e6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5"
Sep 08 14:03:17.770591 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.770559    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8310741a-cff9-4f62-969a-eda2b171a1e6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-x9sk5\" (UID: \"8310741a-cff9-4f62-969a-eda2b171a1e6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5"
Sep 08 14:03:17.770672 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.770634    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhhqd\" (UniqueName: \"kubernetes.io/projected/8310741a-cff9-4f62-969a-eda2b171a1e6-kube-api-access-jhhqd\") pod \"cert-manager-cainjector-559cdd8c87-x9sk5\" (UID: \"8310741a-cff9-4f62-969a-eda2b171a1e6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5"
Sep 08 14:03:17.944654 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:17.944581    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5"
Sep 08 14:03:18.143265 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:18.143233    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl" event={"ID":"8887f740-c623-48b9-9412-44bbdcbb6191","Type":"ContainerStarted","Data":"a57d42a5325c0daebbb696c7aa55acb182c4a8b17739b13eefa31ba77b77e73e"}
Sep 08 14:03:18.213631 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:18.213602    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5"]
Sep 08 14:03:18.216660 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:03:18.216634    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8310741a_cff9_4f62_969a_eda2b171a1e6.slice/crio-ad3add488537a7c88c4f290d9bc5c6d7fd15eb1b68e0da8d3af9ada7935fcbbc WatchSource:0}: Error finding container ad3add488537a7c88c4f290d9bc5c6d7fd15eb1b68e0da8d3af9ada7935fcbbc: Status 404 returned error can't find the container with id ad3add488537a7c88c4f290d9bc5c6d7fd15eb1b68e0da8d3af9ada7935fcbbc
Sep 08 14:03:19.150279 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:19.150218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5" event={"ID":"8310741a-cff9-4f62-969a-eda2b171a1e6","Type":"ContainerStarted","Data":"ad3add488537a7c88c4f290d9bc5c6d7fd15eb1b68e0da8d3af9ada7935fcbbc"}
Sep 08 14:03:22.165688 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:22.165654    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5" event={"ID":"8310741a-cff9-4f62-969a-eda2b171a1e6","Type":"ContainerStarted","Data":"401eb376666bb24dd5df03466f5410e8eb7f1dcf3acc59402cf5b4c2d7c1753b"}
Sep 08 14:03:22.199224 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:22.199172    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x9sk5" podStartSLOduration=2.280956679 podStartE2EDuration="5.199157052s" totalImagesPullingTime="2.918200373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:03:22.19796034 +0000 UTC m=+292.692931021" watchObservedRunningTime="2026-09-08 14:03:22.199157052 +0000 UTC m=+292.694127737"
Sep 08 14:03:24.088590 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.088556    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-857td"]
Sep 08 14:03:24.091462 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.091440    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:24.096939 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.095285    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 14:03:24.096939 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.095890    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tvfsh\""
Sep 08 14:03:24.096939 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.096383    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 14:03:24.096939 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.096444    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 08 14:03:24.096939 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.096745    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 08 14:03:24.110155 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.110129    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-857td"]
Sep 08 14:03:24.113496 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.113475    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-857td\" (UID: \"db4d6ec1-55a6-4913-9387-e2ed8bf28c69\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:24.113602 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.113517    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-857td\" (UID: \"db4d6ec1-55a6-4913-9387-e2ed8bf28c69\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:24.113602 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.113546    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fwdp\" (UniqueName: \"kubernetes.io/projected/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-kube-api-access-7fwdp\") pod \"kuadrant-console-plugin-598f86468b-857td\" (UID: \"db4d6ec1-55a6-4913-9387-e2ed8bf28c69\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:24.214930 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.214897    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fwdp\" (UniqueName: \"kubernetes.io/projected/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-kube-api-access-7fwdp\") pod \"kuadrant-console-plugin-598f86468b-857td\" (UID: \"db4d6ec1-55a6-4913-9387-e2ed8bf28c69\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:24.215107 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.214974    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-857td\" (UID: \"db4d6ec1-55a6-4913-9387-e2ed8bf28c69\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:24.215107 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.215028    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-857td\" (UID: \"db4d6ec1-55a6-4913-9387-e2ed8bf28c69\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:24.215228 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:03:24.215109    2564 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 08 14:03:24.215228 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:03:24.215174    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-plugin-serving-cert podName:db4d6ec1-55a6-4913-9387-e2ed8bf28c69 nodeName:}" failed. No retries permitted until 2026-09-08 14:03:24.715157447 +0000 UTC m=+295.210128109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-plugin-serving-cert") pod "kuadrant-console-plugin-598f86468b-857td" (UID: "db4d6ec1-55a6-4913-9387-e2ed8bf28c69") : secret "plugin-serving-cert" not found
Sep 08 14:03:24.215689 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.215666    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-857td\" (UID: \"db4d6ec1-55a6-4913-9387-e2ed8bf28c69\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:24.240168 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.240144    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fwdp\" (UniqueName: \"kubernetes.io/projected/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-kube-api-access-7fwdp\") pod \"kuadrant-console-plugin-598f86468b-857td\" (UID: \"db4d6ec1-55a6-4913-9387-e2ed8bf28c69\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:24.720177 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.720118    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-857td\" (UID: \"db4d6ec1-55a6-4913-9387-e2ed8bf28c69\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:24.722945 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.722916    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/db4d6ec1-55a6-4913-9387-e2ed8bf28c69-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-857td\" (UID: \"db4d6ec1-55a6-4913-9387-e2ed8bf28c69\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:24.944537 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.944504    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"]
Sep 08 14:03:24.947199 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.947179    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:24.949875 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.949821    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Sep 08 14:03:24.950154 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.950135    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 14:03:24.950228 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.950197    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 14:03:24.950788 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.950771    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Sep 08 14:03:24.951002 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.950983    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Sep 08 14:03:24.951106 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.951010    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-kzwgl\""
Sep 08 14:03:24.979273 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:24.979206    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"]
Sep 08 14:03:25.006535 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.006514    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td"
Sep 08 14:03:25.021889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.021850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0eda0a01-7646-47f1-a4d5-fda4f1248de9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.021984 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.021908    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x4dt\" (UniqueName: \"kubernetes.io/projected/0eda0a01-7646-47f1-a4d5-fda4f1248de9-kube-api-access-5x4dt\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.021984 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.021958    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0eda0a01-7646-47f1-a4d5-fda4f1248de9-tmp\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.022062 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.022021    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0eda0a01-7646-47f1-a4d5-fda4f1248de9-manifests\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.022123 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.022102    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0eda0a01-7646-47f1-a4d5-fda4f1248de9-metrics-cert\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.022167 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.022148    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0eda0a01-7646-47f1-a4d5-fda4f1248de9-webhook-cert\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.123578 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.123547    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0eda0a01-7646-47f1-a4d5-fda4f1248de9-manifests\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.124039 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.123601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0eda0a01-7646-47f1-a4d5-fda4f1248de9-metrics-cert\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.124039 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.123621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0eda0a01-7646-47f1-a4d5-fda4f1248de9-webhook-cert\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.124039 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.123699    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0eda0a01-7646-47f1-a4d5-fda4f1248de9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.124039 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.123752    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x4dt\" (UniqueName: \"kubernetes.io/projected/0eda0a01-7646-47f1-a4d5-fda4f1248de9-kube-api-access-5x4dt\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.124039 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.123837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0eda0a01-7646-47f1-a4d5-fda4f1248de9-tmp\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.124332 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.124081    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0eda0a01-7646-47f1-a4d5-fda4f1248de9-manifests\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.126271 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.126242    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0eda0a01-7646-47f1-a4d5-fda4f1248de9-tmp\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.126470 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.126437    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0eda0a01-7646-47f1-a4d5-fda4f1248de9-metrics-cert\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.126735 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.126716    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0eda0a01-7646-47f1-a4d5-fda4f1248de9-webhook-cert\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.126807 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.126734    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0eda0a01-7646-47f1-a4d5-fda4f1248de9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.135074 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.135054    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x4dt\" (UniqueName: \"kubernetes.io/projected/0eda0a01-7646-47f1-a4d5-fda4f1248de9-kube-api-access-5x4dt\") pod \"opendatahub-operator-controller-manager-77cd5f6c85-k5gsc\" (UID: \"0eda0a01-7646-47f1-a4d5-fda4f1248de9\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.258553 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.258474    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:25.296231 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.296193    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-857td"]
Sep 08 14:03:25.299375 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:03:25.299341    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb4d6ec1_55a6_4913_9387_e2ed8bf28c69.slice/crio-aa5a72164a443d919385fc7cf5de982fe709ae068825e82019dc2bfba1566ad2 WatchSource:0}: Error finding container aa5a72164a443d919385fc7cf5de982fe709ae068825e82019dc2bfba1566ad2: Status 404 returned error can't find the container with id aa5a72164a443d919385fc7cf5de982fe709ae068825e82019dc2bfba1566ad2
Sep 08 14:03:25.546663 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:25.546480    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"]
Sep 08 14:03:25.549481 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:03:25.549451    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eda0a01_7646_47f1_a4d5_fda4f1248de9.slice/crio-706591178bd3c5a70eb30a2f8d25c617d46b9f82c1ac9e7ad2a4ce7a0a3c4169 WatchSource:0}: Error finding container 706591178bd3c5a70eb30a2f8d25c617d46b9f82c1ac9e7ad2a4ce7a0a3c4169: Status 404 returned error can't find the container with id 706591178bd3c5a70eb30a2f8d25c617d46b9f82c1ac9e7ad2a4ce7a0a3c4169
Sep 08 14:03:26.183007 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:26.182975    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc" event={"ID":"0eda0a01-7646-47f1-a4d5-fda4f1248de9","Type":"ContainerStarted","Data":"706591178bd3c5a70eb30a2f8d25c617d46b9f82c1ac9e7ad2a4ce7a0a3c4169"}
Sep 08 14:03:26.184332 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:26.184304    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td" event={"ID":"db4d6ec1-55a6-4913-9387-e2ed8bf28c69","Type":"ContainerStarted","Data":"aa5a72164a443d919385fc7cf5de982fe709ae068825e82019dc2bfba1566ad2"}
Sep 08 14:03:33.765169 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:33.765089    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:03:33.766307 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:33.766286    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:03:33.780888 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:33.780849    2564 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 14:03:34.230221 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:34.230188    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl" event={"ID":"8887f740-c623-48b9-9412-44bbdcbb6191","Type":"ContainerStarted","Data":"ac7ebe96aa3c4c7bb7dabbdd3b9a058bd6e1deee5c0e06cde881ebb93d41ca7d"}
Sep 08 14:03:34.230410 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:34.230328    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl"
Sep 08 14:03:34.231758 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:34.231733    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc" event={"ID":"0eda0a01-7646-47f1-a4d5-fda4f1248de9","Type":"ContainerStarted","Data":"60df5d470690ff29ed4d3c567f6ad540a889f445a70ef4e82856d7d1f8b2da51"}
Sep 08 14:03:34.233451 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:34.233430    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td" event={"ID":"db4d6ec1-55a6-4913-9387-e2ed8bf28c69","Type":"ContainerStarted","Data":"aca0ea2b45c607f2ef0b44ba8bb48eb71395512357c5c61aec596ae7a86aab14"}
Sep 08 14:03:34.272475 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:34.272433    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl" podStartSLOduration=2.057066701 podStartE2EDuration="18.272421555s" totalImagesPullingTime="16.215354854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:03:34.2700434 +0000 UTC m=+304.765014081" watchObservedRunningTime="2026-09-08 14:03:34.272421555 +0000 UTC m=+304.767392239"
Sep 08 14:03:34.366385 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:34.366335    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-857td" podStartSLOduration=1.9142028720000002 podStartE2EDuration="10.366319489s" totalImagesPullingTime="8.452116617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:03:34.313981642 +0000 UTC m=+304.808952326" watchObservedRunningTime="2026-09-08 14:03:34.366319489 +0000 UTC m=+304.861290198"
Sep 08 14:03:35.239618 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:35.239579    2564 generic.go:356] "Generic (PLEG): container finished" podID="0eda0a01-7646-47f1-a4d5-fda4f1248de9" containerID="60df5d470690ff29ed4d3c567f6ad540a889f445a70ef4e82856d7d1f8b2da51" exitCode=0
Sep 08 14:03:35.240011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:35.239666    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc" event={"ID":"0eda0a01-7646-47f1-a4d5-fda4f1248de9","Type":"ContainerDied","Data":"60df5d470690ff29ed4d3c567f6ad540a889f445a70ef4e82856d7d1f8b2da51"}
Sep 08 14:03:35.240348 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:35.240328    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:03:39.257909 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:39.257848    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc" event={"ID":"0eda0a01-7646-47f1-a4d5-fda4f1248de9","Type":"ContainerStarted","Data":"605c21a30f9fea5a04c80d00e60f71e2ec518f25cacd969a683ac203b1927684"}
Sep 08 14:03:39.258295 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:39.257929    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:03:39.283272 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:39.283226    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc" podStartSLOduration=2.9003560029999997 podStartE2EDuration="15.283209767s" totalImagesPullingTime="11.810254477s" totalInitContainerRuntime="572.599287ms" isStatefulPod=false podCreationTimestamp="2026-09-08 14:03:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:03:39.282107399 +0000 UTC m=+309.777078084" watchObservedRunningTime="2026-09-08 14:03:39.283209767 +0000 UTC m=+309.778180450"
Sep 08 14:03:40.243212 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:40.243180    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-s9knl"
Sep 08 14:03:50.264291 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:03:50.264259    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-77cd5f6c85-k5gsc"
Sep 08 14:04:00.646846 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.646813    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"]
Sep 08 14:04:00.654672 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.654652    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:00.659677 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.659648    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 08 14:04:00.659899 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.659883    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 08 14:04:00.660114 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.660094    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 14:04:00.660303 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.660287    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 14:04:00.660609 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.660585    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qfj4b\""
Sep 08 14:04:00.661292 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.661269    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"]
Sep 08 14:04:00.745259 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.745225    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xll6k\" (UniqueName: \"kubernetes.io/projected/ad87da59-0feb-4378-a627-f2a2db2e6975-kube-api-access-xll6k\") pod \"kube-auth-proxy-58f4c5d6b6-rrbxj\" (UID: \"ad87da59-0feb-4378-a627-f2a2db2e6975\") " pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:00.745437 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.745276    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad87da59-0feb-4378-a627-f2a2db2e6975-tmp\") pod \"kube-auth-proxy-58f4c5d6b6-rrbxj\" (UID: \"ad87da59-0feb-4378-a627-f2a2db2e6975\") " pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:00.745437 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.745365    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad87da59-0feb-4378-a627-f2a2db2e6975-tls-certs\") pod \"kube-auth-proxy-58f4c5d6b6-rrbxj\" (UID: \"ad87da59-0feb-4378-a627-f2a2db2e6975\") " pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:00.846081 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.846040    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad87da59-0feb-4378-a627-f2a2db2e6975-tls-certs\") pod \"kube-auth-proxy-58f4c5d6b6-rrbxj\" (UID: \"ad87da59-0feb-4378-a627-f2a2db2e6975\") " pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:00.846292 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.846113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xll6k\" (UniqueName: \"kubernetes.io/projected/ad87da59-0feb-4378-a627-f2a2db2e6975-kube-api-access-xll6k\") pod \"kube-auth-proxy-58f4c5d6b6-rrbxj\" (UID: \"ad87da59-0feb-4378-a627-f2a2db2e6975\") " pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:00.846292 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.846147    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad87da59-0feb-4378-a627-f2a2db2e6975-tmp\") pod \"kube-auth-proxy-58f4c5d6b6-rrbxj\" (UID: \"ad87da59-0feb-4378-a627-f2a2db2e6975\") " pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:00.846292 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:04:00.846224    2564 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 08 14:04:00.846460 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:04:00.846312    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad87da59-0feb-4378-a627-f2a2db2e6975-tls-certs podName:ad87da59-0feb-4378-a627-f2a2db2e6975 nodeName:}" failed. No retries permitted until 2026-09-08 14:04:01.34628817 +0000 UTC m=+331.841258833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ad87da59-0feb-4378-a627-f2a2db2e6975-tls-certs") pod "kube-auth-proxy-58f4c5d6b6-rrbxj" (UID: "ad87da59-0feb-4378-a627-f2a2db2e6975") : secret "kube-auth-proxy-tls" not found
Sep 08 14:04:00.848583 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.848556    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad87da59-0feb-4378-a627-f2a2db2e6975-tmp\") pod \"kube-auth-proxy-58f4c5d6b6-rrbxj\" (UID: \"ad87da59-0feb-4378-a627-f2a2db2e6975\") " pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:00.858391 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:00.858365    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xll6k\" (UniqueName: \"kubernetes.io/projected/ad87da59-0feb-4378-a627-f2a2db2e6975-kube-api-access-xll6k\") pod \"kube-auth-proxy-58f4c5d6b6-rrbxj\" (UID: \"ad87da59-0feb-4378-a627-f2a2db2e6975\") " pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:01.350623 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:01.350577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad87da59-0feb-4378-a627-f2a2db2e6975-tls-certs\") pod \"kube-auth-proxy-58f4c5d6b6-rrbxj\" (UID: \"ad87da59-0feb-4378-a627-f2a2db2e6975\") " pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:01.353347 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:01.353307    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad87da59-0feb-4378-a627-f2a2db2e6975-tls-certs\") pod \"kube-auth-proxy-58f4c5d6b6-rrbxj\" (UID: \"ad87da59-0feb-4378-a627-f2a2db2e6975\") " pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:01.567881 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:01.567831    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"
Sep 08 14:04:01.851091 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:01.851061    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj"]
Sep 08 14:04:01.854497 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:04:01.854464    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad87da59_0feb_4378_a627_f2a2db2e6975.slice/crio-5b524b6bc550ed5d0e83b9802c0fa5bd8513a85564a55bd4eee3c56ab0f81361 WatchSource:0}: Error finding container 5b524b6bc550ed5d0e83b9802c0fa5bd8513a85564a55bd4eee3c56ab0f81361: Status 404 returned error can't find the container with id 5b524b6bc550ed5d0e83b9802c0fa5bd8513a85564a55bd4eee3c56ab0f81361
Sep 08 14:04:02.361020 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:02.360980    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj" event={"ID":"ad87da59-0feb-4378-a627-f2a2db2e6975","Type":"ContainerStarted","Data":"5b524b6bc550ed5d0e83b9802c0fa5bd8513a85564a55bd4eee3c56ab0f81361"}
Sep 08 14:04:03.976995 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:03.976956    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"]
Sep 08 14:04:03.979224 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:03.979204    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:03.984254 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:03.984229    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Sep 08 14:04:03.984487 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:03.984470    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Sep 08 14:04:03.985176 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:03.985157    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Sep 08 14:04:03.985303 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:03.985202    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-t47hq\""
Sep 08 14:04:03.985433 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:03.985405    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Sep 08 14:04:03.986041 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:03.986031    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Sep 08 14:04:04.005981 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.005949    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"]
Sep 08 14:04:04.076757 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.076726    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d5m9\" (UniqueName: \"kubernetes.io/projected/f129a405-b420-47b4-bb75-4bdc5c5580b6-kube-api-access-4d5m9\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.076757 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.076761    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f129a405-b420-47b4-bb75-4bdc5c5580b6-metrics-certs\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.077009 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.076798    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f129a405-b420-47b4-bb75-4bdc5c5580b6-manager-config\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.077009 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.076912    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f129a405-b420-47b4-bb75-4bdc5c5580b6-cert\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.178202 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.178123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f129a405-b420-47b4-bb75-4bdc5c5580b6-manager-config\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.178202 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.178184    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f129a405-b420-47b4-bb75-4bdc5c5580b6-cert\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.178421 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.178227    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d5m9\" (UniqueName: \"kubernetes.io/projected/f129a405-b420-47b4-bb75-4bdc5c5580b6-kube-api-access-4d5m9\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.178421 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.178248    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f129a405-b420-47b4-bb75-4bdc5c5580b6-metrics-certs\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.178815 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.178787    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f129a405-b420-47b4-bb75-4bdc5c5580b6-manager-config\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.180882 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.180842    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f129a405-b420-47b4-bb75-4bdc5c5580b6-metrics-certs\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.181014 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.180996    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f129a405-b420-47b4-bb75-4bdc5c5580b6-cert\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.188464 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.188441    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d5m9\" (UniqueName: \"kubernetes.io/projected/f129a405-b420-47b4-bb75-4bdc5c5580b6-kube-api-access-4d5m9\") pod \"jobset-controller-manager-c64769648-gxrfh\" (UID: \"f129a405-b420-47b4-bb75-4bdc5c5580b6\") " pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.289075 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.289038    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:04.372399 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.372363    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj" event={"ID":"ad87da59-0feb-4378-a627-f2a2db2e6975","Type":"ContainerStarted","Data":"10e398c8959da692e9ced19d85e63a75899288558a76dc36c97bdbb95f68eab2"}
Sep 08 14:04:04.391505 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.391451    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58f4c5d6b6-rrbxj" podStartSLOduration=2.388444024 podStartE2EDuration="4.391434665s" totalImagesPullingTime="2.002990641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:04:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:04:04.390037798 +0000 UTC m=+334.885008484" watchObservedRunningTime="2026-09-08 14:04:04.391434665 +0000 UTC m=+334.886405401"
Sep 08 14:04:04.560621 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:04.560585    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"]
Sep 08 14:04:04.562656 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:04:04.562630    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf129a405_b420_47b4_bb75_4bdc5c5580b6.slice/crio-1bfc9607e71b4d02b653a1ecb53b4d3868788edde32dcec8b880b4019c4841c6 WatchSource:0}: Error finding container 1bfc9607e71b4d02b653a1ecb53b4d3868788edde32dcec8b880b4019c4841c6: Status 404 returned error can't find the container with id 1bfc9607e71b4d02b653a1ecb53b4d3868788edde32dcec8b880b4019c4841c6
Sep 08 14:04:05.378115 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:05.378076    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh" event={"ID":"f129a405-b420-47b4-bb75-4bdc5c5580b6","Type":"ContainerStarted","Data":"1bfc9607e71b4d02b653a1ecb53b4d3868788edde32dcec8b880b4019c4841c6"}
Sep 08 14:04:07.390320 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:07.390279    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh" event={"ID":"f129a405-b420-47b4-bb75-4bdc5c5580b6","Type":"ContainerStarted","Data":"428a304456a138110f1719270ca58c9b6801e67a09cc269dfcf30064a37c20e2"}
Sep 08 14:04:07.390906 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:07.390364    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:07.408752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:07.408705    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh" podStartSLOduration=1.744588309 podStartE2EDuration="4.408690004s" totalImagesPullingTime="2.664101695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:04:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:04:07.406990824 +0000 UTC m=+337.901961510" watchObservedRunningTime="2026-09-08 14:04:07.408690004 +0000 UTC m=+337.903660687"
Sep 08 14:04:18.400483 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:18.400454    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-c64769648-gxrfh"
Sep 08 14:04:19.917196 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:19.917161    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7848477867-b679m"]
Sep 08 14:04:19.919465 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:19.919447    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:19.921814 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:19.921775    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 14:04:19.921980 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:19.921885    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vj9d6\""
Sep 08 14:04:19.922909 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:19.922891    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 14:04:19.923053 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:19.922891    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 14:04:19.923158 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:19.922902    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 14:04:19.923239 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:19.922909    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 14:04:19.931016 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:19.930994    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7848477867-b679m"]
Sep 08 14:04:20.028996 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.028948    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-webhook-cert\") pod \"maas-controller-7848477867-b679m\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") " pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:20.028996 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.029001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-metrics-certs\") pod \"maas-controller-7848477867-b679m\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") " pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:20.029201 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.029154    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfvrr\" (UniqueName: \"kubernetes.io/projected/028e0d72-7ea2-46d4-b7db-df107d0b9698-kube-api-access-lfvrr\") pod \"maas-controller-7848477867-b679m\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") " pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:20.103895 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.103846    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-l49z4"]
Sep 08 14:04:20.106089 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.106070    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:20.120722 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.120697    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-l49z4"]
Sep 08 14:04:20.129900 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.129875    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfvrr\" (UniqueName: \"kubernetes.io/projected/028e0d72-7ea2-46d4-b7db-df107d0b9698-kube-api-access-lfvrr\") pod \"maas-controller-7848477867-b679m\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") " pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:20.130024 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.129948    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-webhook-cert\") pod \"maas-controller-7848477867-b679m\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") " pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:20.130024 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.129976    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-metrics-certs\") pod \"maas-controller-7848477867-b679m\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") " pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:20.130136 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:04:20.130091    2564 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 08 14:04:20.130187 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:04:20.130165    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-webhook-cert podName:028e0d72-7ea2-46d4-b7db-df107d0b9698 nodeName:}" failed. No retries permitted until 2026-09-08 14:04:20.630148202 +0000 UTC m=+351.125118864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-webhook-cert") pod "maas-controller-7848477867-b679m" (UID: "028e0d72-7ea2-46d4-b7db-df107d0b9698") : secret "maas-controller-webhook-cert" not found
Sep 08 14:04:20.132640 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.132618    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-metrics-certs\") pod \"maas-controller-7848477867-b679m\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") " pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:20.142401 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.142377    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfvrr\" (UniqueName: \"kubernetes.io/projected/028e0d72-7ea2-46d4-b7db-df107d0b9698-kube-api-access-lfvrr\") pod \"maas-controller-7848477867-b679m\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") " pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:20.230635 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.230606    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-metrics-certs\") pod \"maas-controller-6d9494c8c4-l49z4\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") " pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:20.230762 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.230664    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-webhook-cert\") pod \"maas-controller-6d9494c8c4-l49z4\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") " pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:20.230802 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.230753    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsxcd\" (UniqueName: \"kubernetes.io/projected/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-kube-api-access-jsxcd\") pod \"maas-controller-6d9494c8c4-l49z4\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") " pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:20.331787 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.331756    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsxcd\" (UniqueName: \"kubernetes.io/projected/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-kube-api-access-jsxcd\") pod \"maas-controller-6d9494c8c4-l49z4\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") " pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:20.331975 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.331817    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-metrics-certs\") pod \"maas-controller-6d9494c8c4-l49z4\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") " pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:20.331975 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.331859    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-webhook-cert\") pod \"maas-controller-6d9494c8c4-l49z4\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") " pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:20.332126 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:04:20.331992    2564 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 08 14:04:20.332126 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:04:20.332059    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-webhook-cert podName:f2f2f32a-8fe4-4408-8b0a-0c100b1217fc nodeName:}" failed. No retries permitted until 2026-09-08 14:04:20.832045506 +0000 UTC m=+351.327016169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-webhook-cert") pod "maas-controller-6d9494c8c4-l49z4" (UID: "f2f2f32a-8fe4-4408-8b0a-0c100b1217fc") : secret "maas-controller-webhook-cert" not found
Sep 08 14:04:20.334377 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.334357    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-metrics-certs\") pod \"maas-controller-6d9494c8c4-l49z4\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") " pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:20.340746 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.340728    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsxcd\" (UniqueName: \"kubernetes.io/projected/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-kube-api-access-jsxcd\") pod \"maas-controller-6d9494c8c4-l49z4\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") " pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:20.635239 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.635160    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-webhook-cert\") pod \"maas-controller-7848477867-b679m\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") " pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:20.637612 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.637592    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-webhook-cert\") pod \"maas-controller-7848477867-b679m\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") " pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:20.832523 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.832475    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:20.837644 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.837615    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-webhook-cert\") pod \"maas-controller-6d9494c8c4-l49z4\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") " pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:20.840166 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:20.840144    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-webhook-cert\") pod \"maas-controller-6d9494c8c4-l49z4\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") " pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:21.016322 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:21.016295    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:21.105207 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:21.105175    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7848477867-b679m"]
Sep 08 14:04:21.106629 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:04:21.106599    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod028e0d72_7ea2_46d4_b7db_df107d0b9698.slice/crio-d7a1cea165dc0781da1a8323f30249e2e07e0a3c7297aed4e561367386620391 WatchSource:0}: Error finding container d7a1cea165dc0781da1a8323f30249e2e07e0a3c7297aed4e561367386620391: Status 404 returned error can't find the container with id d7a1cea165dc0781da1a8323f30249e2e07e0a3c7297aed4e561367386620391
Sep 08 14:04:21.286501 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:21.286426    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-l49z4"]
Sep 08 14:04:21.289166 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:04:21.289137    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2f2f32a_8fe4_4408_8b0a_0c100b1217fc.slice/crio-1261b8876478638facff69d95942c918fcb347558b0eef40d99927b975f66dbf WatchSource:0}: Error finding container 1261b8876478638facff69d95942c918fcb347558b0eef40d99927b975f66dbf: Status 404 returned error can't find the container with id 1261b8876478638facff69d95942c918fcb347558b0eef40d99927b975f66dbf
Sep 08 14:04:21.456458 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:21.456421    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7848477867-b679m" event={"ID":"028e0d72-7ea2-46d4-b7db-df107d0b9698","Type":"ContainerStarted","Data":"d7a1cea165dc0781da1a8323f30249e2e07e0a3c7297aed4e561367386620391"}
Sep 08 14:04:21.457622 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:21.457593    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9494c8c4-l49z4" event={"ID":"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc","Type":"ContainerStarted","Data":"1261b8876478638facff69d95942c918fcb347558b0eef40d99927b975f66dbf"}
Sep 08 14:04:23.468807 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:23.468694    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7848477867-b679m" event={"ID":"028e0d72-7ea2-46d4-b7db-df107d0b9698","Type":"ContainerStarted","Data":"02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc"}
Sep 08 14:04:23.468807 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:23.468827    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:23.470344 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:23.470321    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9494c8c4-l49z4" event={"ID":"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc","Type":"ContainerStarted","Data":"56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715"}
Sep 08 14:04:23.470444 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:23.470376    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:23.489734 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:23.489680    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7848477867-b679m" podStartSLOduration=2.77044053 podStartE2EDuration="4.489661644s" totalImagesPullingTime="1.719221114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:04:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:04:23.489302506 +0000 UTC m=+353.984273192" watchObservedRunningTime="2026-09-08 14:04:23.489661644 +0000 UTC m=+353.984632330"
Sep 08 14:04:23.515161 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:23.515120    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d9494c8c4-l49z4" podStartSLOduration=1.98198485 podStartE2EDuration="3.515105524s" totalImagesPullingTime="1.533120674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:04:23.513296651 +0000 UTC m=+354.008267360" watchObservedRunningTime="2026-09-08 14:04:23.515105524 +0000 UTC m=+354.010076207"
Sep 08 14:04:34.477638 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.477606    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:34.478138 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.477688    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:04:34.551679 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.551645    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7848477867-b679m"]
Sep 08 14:04:34.552007 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.551946    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7848477867-b679m" podUID="028e0d72-7ea2-46d4-b7db-df107d0b9698" containerName="manager" containerID="cri-o://02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc" gracePeriod=30
Sep 08 14:04:34.797168 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.797146    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:34.867291 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.867261    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-metrics-certs\") pod \"028e0d72-7ea2-46d4-b7db-df107d0b9698\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") "
Sep 08 14:04:34.867459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.867301    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-webhook-cert\") pod \"028e0d72-7ea2-46d4-b7db-df107d0b9698\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") "
Sep 08 14:04:34.867459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.867321    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfvrr\" (UniqueName: \"kubernetes.io/projected/028e0d72-7ea2-46d4-b7db-df107d0b9698-kube-api-access-lfvrr\") pod \"028e0d72-7ea2-46d4-b7db-df107d0b9698\" (UID: \"028e0d72-7ea2-46d4-b7db-df107d0b9698\") "
Sep 08 14:04:34.869662 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.869632    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "028e0d72-7ea2-46d4-b7db-df107d0b9698" (UID: "028e0d72-7ea2-46d4-b7db-df107d0b9698"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:04:34.869662 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.869646    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "028e0d72-7ea2-46d4-b7db-df107d0b9698" (UID: "028e0d72-7ea2-46d4-b7db-df107d0b9698"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:04:34.869845 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.869665    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/028e0d72-7ea2-46d4-b7db-df107d0b9698-kube-api-access-lfvrr" (OuterVolumeSpecName: "kube-api-access-lfvrr") pod "028e0d72-7ea2-46d4-b7db-df107d0b9698" (UID: "028e0d72-7ea2-46d4-b7db-df107d0b9698"). InnerVolumeSpecName "kube-api-access-lfvrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:04:34.968664 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.968636    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-webhook-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:04:34.968664 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.968662    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfvrr\" (UniqueName: \"kubernetes.io/projected/028e0d72-7ea2-46d4-b7db-df107d0b9698-kube-api-access-lfvrr\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:04:34.968664 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:34.968671    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/028e0d72-7ea2-46d4-b7db-df107d0b9698-metrics-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:04:35.524409 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:35.524375    2564 generic.go:356] "Generic (PLEG): container finished" podID="028e0d72-7ea2-46d4-b7db-df107d0b9698" containerID="02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc" exitCode=0
Sep 08 14:04:35.524845 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:35.524421    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7848477867-b679m"
Sep 08 14:04:35.524845 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:35.524463    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7848477867-b679m" event={"ID":"028e0d72-7ea2-46d4-b7db-df107d0b9698","Type":"ContainerDied","Data":"02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc"}
Sep 08 14:04:35.524845 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:35.524514    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7848477867-b679m" event={"ID":"028e0d72-7ea2-46d4-b7db-df107d0b9698","Type":"ContainerDied","Data":"d7a1cea165dc0781da1a8323f30249e2e07e0a3c7297aed4e561367386620391"}
Sep 08 14:04:35.524845 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:35.524537    2564 scope.go:117] "RemoveContainer" containerID="02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc"
Sep 08 14:04:35.533545 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:35.533527    2564 scope.go:117] "RemoveContainer" containerID="02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc"
Sep 08 14:04:35.533839 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:04:35.533821    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc\": container with ID starting with 02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc not found: ID does not exist" containerID="02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc"
Sep 08 14:04:35.533931 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:35.533847    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc"} err="failed to get container status \"02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc\": rpc error: code = NotFound desc = could not find container \"02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc\": container with ID starting with 02e7368d3e3ff81d692f4b89127cfb36a2bd3d3c67a975ac5fde857e40694cfc not found: ID does not exist"
Sep 08 14:04:35.552670 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:35.552642    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7848477867-b679m"]
Sep 08 14:04:35.554087 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:35.554068    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7848477867-b679m"]
Sep 08 14:04:35.958155 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:35.958069    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="028e0d72-7ea2-46d4-b7db-df107d0b9698" path="/var/lib/kubelet/pods/028e0d72-7ea2-46d4-b7db-df107d0b9698/volumes"
Sep 08 14:04:59.166683 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.166649    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"]
Sep 08 14:04:59.167190 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.166989    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="028e0d72-7ea2-46d4-b7db-df107d0b9698" containerName="manager"
Sep 08 14:04:59.167190 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.167003    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="028e0d72-7ea2-46d4-b7db-df107d0b9698" containerName="manager"
Sep 08 14:04:59.167190 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.167066    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="028e0d72-7ea2-46d4-b7db-df107d0b9698" containerName="manager"
Sep 08 14:04:59.172219 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.172199    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"
Sep 08 14:04:59.175958 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.175937    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 08 14:04:59.176098 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.176035    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-4x9x8\""
Sep 08 14:04:59.193624 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.193596    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"]
Sep 08 14:04:59.266032 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.265999    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-bb7b75cc4-k9sn2"]
Sep 08 14:04:59.269683 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.269661    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:04:59.272426 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.272399    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 08 14:04:59.277526 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.277504    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e03268be-cb29-4268-895d-1b91e5e17919-cert\") pod \"llmisvc-controller-manager-7d548556bc-8rrsf\" (UID: \"e03268be-cb29-4268-895d-1b91e5e17919\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"
Sep 08 14:04:59.277633 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.277552    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m4s5\" (UniqueName: \"kubernetes.io/projected/e03268be-cb29-4268-895d-1b91e5e17919-kube-api-access-8m4s5\") pod \"llmisvc-controller-manager-7d548556bc-8rrsf\" (UID: \"e03268be-cb29-4268-895d-1b91e5e17919\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"
Sep 08 14:04:59.281152 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.281129    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mz2v5\""
Sep 08 14:04:59.288976 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.288956    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Sep 08 14:04:59.291072 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.291052    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-bb7b75cc4-k9sn2"]
Sep 08 14:04:59.378261 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.378230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m4s5\" (UniqueName: \"kubernetes.io/projected/e03268be-cb29-4268-895d-1b91e5e17919-kube-api-access-8m4s5\") pod \"llmisvc-controller-manager-7d548556bc-8rrsf\" (UID: \"e03268be-cb29-4268-895d-1b91e5e17919\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"
Sep 08 14:04:59.378415 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.378292    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99cb8a91-5ddb-4e51-896c-769517de4696-metrics-certs\") pod \"odh-model-controller-bb7b75cc4-k9sn2\" (UID: \"99cb8a91-5ddb-4e51-896c-769517de4696\") " pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:04:59.378415 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.378323    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wldms\" (UniqueName: \"kubernetes.io/projected/99cb8a91-5ddb-4e51-896c-769517de4696-kube-api-access-wldms\") pod \"odh-model-controller-bb7b75cc4-k9sn2\" (UID: \"99cb8a91-5ddb-4e51-896c-769517de4696\") " pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:04:59.378415 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.378364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99cb8a91-5ddb-4e51-896c-769517de4696-cert\") pod \"odh-model-controller-bb7b75cc4-k9sn2\" (UID: \"99cb8a91-5ddb-4e51-896c-769517de4696\") " pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:04:59.378543 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.378499    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e03268be-cb29-4268-895d-1b91e5e17919-cert\") pod \"llmisvc-controller-manager-7d548556bc-8rrsf\" (UID: \"e03268be-cb29-4268-895d-1b91e5e17919\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"
Sep 08 14:04:59.378580 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:04:59.378564    2564 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 08 14:04:59.378626 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:04:59.378616    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e03268be-cb29-4268-895d-1b91e5e17919-cert podName:e03268be-cb29-4268-895d-1b91e5e17919 nodeName:}" failed. No retries permitted until 2026-09-08 14:04:59.878601656 +0000 UTC m=+390.373572319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e03268be-cb29-4268-895d-1b91e5e17919-cert") pod "llmisvc-controller-manager-7d548556bc-8rrsf" (UID: "e03268be-cb29-4268-895d-1b91e5e17919") : secret "llmisvc-webhook-server-cert" not found
Sep 08 14:04:59.436289 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.436230    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m4s5\" (UniqueName: \"kubernetes.io/projected/e03268be-cb29-4268-895d-1b91e5e17919-kube-api-access-8m4s5\") pod \"llmisvc-controller-manager-7d548556bc-8rrsf\" (UID: \"e03268be-cb29-4268-895d-1b91e5e17919\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"
Sep 08 14:04:59.479388 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.479346    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99cb8a91-5ddb-4e51-896c-769517de4696-metrics-certs\") pod \"odh-model-controller-bb7b75cc4-k9sn2\" (UID: \"99cb8a91-5ddb-4e51-896c-769517de4696\") " pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:04:59.479388 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.479385    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wldms\" (UniqueName: \"kubernetes.io/projected/99cb8a91-5ddb-4e51-896c-769517de4696-kube-api-access-wldms\") pod \"odh-model-controller-bb7b75cc4-k9sn2\" (UID: \"99cb8a91-5ddb-4e51-896c-769517de4696\") " pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:04:59.479626 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.479421    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99cb8a91-5ddb-4e51-896c-769517de4696-cert\") pod \"odh-model-controller-bb7b75cc4-k9sn2\" (UID: \"99cb8a91-5ddb-4e51-896c-769517de4696\") " pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:04:59.479626 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:04:59.479518    2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-metrics-tls: secret "odh-model-controller-metrics-tls" not found
Sep 08 14:04:59.479626 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:04:59.479593    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99cb8a91-5ddb-4e51-896c-769517de4696-metrics-certs podName:99cb8a91-5ddb-4e51-896c-769517de4696 nodeName:}" failed. No retries permitted until 2026-09-08 14:04:59.979572348 +0000 UTC m=+390.474543024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99cb8a91-5ddb-4e51-896c-769517de4696-metrics-certs") pod "odh-model-controller-bb7b75cc4-k9sn2" (UID: "99cb8a91-5ddb-4e51-896c-769517de4696") : secret "odh-model-controller-metrics-tls" not found
Sep 08 14:04:59.482145 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.482119    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99cb8a91-5ddb-4e51-896c-769517de4696-cert\") pod \"odh-model-controller-bb7b75cc4-k9sn2\" (UID: \"99cb8a91-5ddb-4e51-896c-769517de4696\") " pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:04:59.500046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.500024    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wldms\" (UniqueName: \"kubernetes.io/projected/99cb8a91-5ddb-4e51-896c-769517de4696-kube-api-access-wldms\") pod \"odh-model-controller-bb7b75cc4-k9sn2\" (UID: \"99cb8a91-5ddb-4e51-896c-769517de4696\") " pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:04:59.884319 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.884284    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e03268be-cb29-4268-895d-1b91e5e17919-cert\") pod \"llmisvc-controller-manager-7d548556bc-8rrsf\" (UID: \"e03268be-cb29-4268-895d-1b91e5e17919\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"
Sep 08 14:04:59.887004 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.886977    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e03268be-cb29-4268-895d-1b91e5e17919-cert\") pod \"llmisvc-controller-manager-7d548556bc-8rrsf\" (UID: \"e03268be-cb29-4268-895d-1b91e5e17919\") " pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"
Sep 08 14:04:59.985396 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.985364    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99cb8a91-5ddb-4e51-896c-769517de4696-metrics-certs\") pod \"odh-model-controller-bb7b75cc4-k9sn2\" (UID: \"99cb8a91-5ddb-4e51-896c-769517de4696\") " pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:04:59.987938 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:04:59.987915    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99cb8a91-5ddb-4e51-896c-769517de4696-metrics-certs\") pod \"odh-model-controller-bb7b75cc4-k9sn2\" (UID: \"99cb8a91-5ddb-4e51-896c-769517de4696\") " pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:05:00.082761 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:00.082729    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"
Sep 08 14:05:00.182706 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:00.182652    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:05:00.357735 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:00.357715    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"]
Sep 08 14:05:00.359877 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:05:00.359833    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode03268be_cb29_4268_895d_1b91e5e17919.slice/crio-f632a6ccac2c9e749ddb4841586a35e9167c2d9086e055e1cc690cb245c445bd WatchSource:0}: Error finding container f632a6ccac2c9e749ddb4841586a35e9167c2d9086e055e1cc690cb245c445bd: Status 404 returned error can't find the container with id f632a6ccac2c9e749ddb4841586a35e9167c2d9086e055e1cc690cb245c445bd
Sep 08 14:05:00.459853 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:00.459829    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-bb7b75cc4-k9sn2"]
Sep 08 14:05:00.461251 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:05:00.461220    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cb8a91_5ddb_4e51_896c_769517de4696.slice/crio-928e470f06257ead1de0f64122400fa12dbbca4c4f0a4ab4f07b44b775c90901 WatchSource:0}: Error finding container 928e470f06257ead1de0f64122400fa12dbbca4c4f0a4ab4f07b44b775c90901: Status 404 returned error can't find the container with id 928e470f06257ead1de0f64122400fa12dbbca4c4f0a4ab4f07b44b775c90901
Sep 08 14:05:00.639298 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:00.639258    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf" event={"ID":"e03268be-cb29-4268-895d-1b91e5e17919","Type":"ContainerStarted","Data":"f632a6ccac2c9e749ddb4841586a35e9167c2d9086e055e1cc690cb245c445bd"}
Sep 08 14:05:00.640494 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:00.640473    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2" event={"ID":"99cb8a91-5ddb-4e51-896c-769517de4696","Type":"ContainerStarted","Data":"928e470f06257ead1de0f64122400fa12dbbca4c4f0a4ab4f07b44b775c90901"}
Sep 08 14:05:04.663807 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:04.663704    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf" event={"ID":"e03268be-cb29-4268-895d-1b91e5e17919","Type":"ContainerStarted","Data":"23b928d11e20ea5fdaa4a6ceee27b9b4e7448f8e5bf1e730267b651683748fb2"}
Sep 08 14:05:04.664235 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:04.663815    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"
Sep 08 14:05:04.665366 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:04.665339    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2" event={"ID":"99cb8a91-5ddb-4e51-896c-769517de4696","Type":"ContainerStarted","Data":"2b28ed10b45f22655a88267ff21039a13be68a8bf13e7dd74d8b91403059a556"}
Sep 08 14:05:04.665483 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:04.665473    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:05:04.684333 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:04.684294    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf" podStartSLOduration=1.711100794 podStartE2EDuration="5.684280974s" totalImagesPullingTime="3.97318018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:05:04.683436918 +0000 UTC m=+395.178407603" watchObservedRunningTime="2026-09-08 14:05:04.684280974 +0000 UTC m=+395.179251658"
Sep 08 14:05:04.703176 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:04.703115    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2" podStartSLOduration=2.4956288779999998 podStartE2EDuration="5.703087504s" totalImagesPullingTime="3.207458626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:05:04.701217889 +0000 UTC m=+395.196188572" watchObservedRunningTime="2026-09-08 14:05:04.703087504 +0000 UTC m=+395.198058189"
Sep 08 14:05:15.672821 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:15.672789    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-bb7b75cc4-k9sn2"
Sep 08 14:05:35.672969 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:05:35.672934    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-7d548556bc-8rrsf"
Sep 08 14:06:49.076710 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.076678    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"]
Sep 08 14:06:49.079462 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.079440    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.082021 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.081963    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 14:06:49.082201 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.082174    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-b2v2m\""
Sep 08 14:06:49.082991 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.082971    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 14:06:49.083064 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.082971    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 08 14:06:49.088891 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.088850    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"]
Sep 08 14:06:49.186097 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.186063    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-home\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.186275 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.186103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26706892-e4b5-45b4-b087-61802acfbfc2-tls-certs\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.186275 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.186137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-model-cache\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.186275 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.186223    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-dshm\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.186275 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.186252    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl2pb\" (UniqueName: \"kubernetes.io/projected/26706892-e4b5-45b4-b087-61802acfbfc2-kube-api-access-nl2pb\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.186446 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.186301    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-tmp-dir\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.287085 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.287052    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-home\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.287270 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.287096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26706892-e4b5-45b4-b087-61802acfbfc2-tls-certs\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.287270 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.287130    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-model-cache\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.287270 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.287163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-dshm\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.287270 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.287179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl2pb\" (UniqueName: \"kubernetes.io/projected/26706892-e4b5-45b4-b087-61802acfbfc2-kube-api-access-nl2pb\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.287270 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.287229    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-tmp-dir\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.287556 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.287481    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-home\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.287556 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.287523    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-model-cache\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.287644 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.287604    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-tmp-dir\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.289565 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.289543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26706892-e4b5-45b4-b087-61802acfbfc2-dshm\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.289809 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.289791    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26706892-e4b5-45b4-b087-61802acfbfc2-tls-certs\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.296274 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.296249    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl2pb\" (UniqueName: \"kubernetes.io/projected/26706892-e4b5-45b4-b087-61802acfbfc2-kube-api-access-nl2pb\") pod \"e2e-embedding-simulated-kserve-698db7d589-qm6px\" (UID: \"26706892-e4b5-45b4-b087-61802acfbfc2\") " pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.389532 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.389456    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:49.661231 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:49.661203    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"]
Sep 08 14:06:49.663930 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:06:49.663858    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26706892_e4b5_45b4_b087_61802acfbfc2.slice/crio-6287fe87a82a2d3f2635e0f8799e00a10b24ef1013d610b3dc7e9ec7b83aafc6 WatchSource:0}: Error finding container 6287fe87a82a2d3f2635e0f8799e00a10b24ef1013d610b3dc7e9ec7b83aafc6: Status 404 returned error can't find the container with id 6287fe87a82a2d3f2635e0f8799e00a10b24ef1013d610b3dc7e9ec7b83aafc6
Sep 08 14:06:50.157452 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:50.157399    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px" event={"ID":"26706892-e4b5-45b4-b087-61802acfbfc2","Type":"ContainerStarted","Data":"6287fe87a82a2d3f2635e0f8799e00a10b24ef1013d610b3dc7e9ec7b83aafc6"}
Sep 08 14:06:51.164653 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:51.164610    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px" event={"ID":"26706892-e4b5-45b4-b087-61802acfbfc2","Type":"ContainerStarted","Data":"6b93fd7b4676f82a550dd6aa9dbf75898f90c1627249d602cef838b69fce7b88"}
Sep 08 14:06:51.184765 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:51.184714    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px" podStartSLOduration=1.162655928 podStartE2EDuration="2.184699432s" totalImagesPullingTime="1.022043504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:06:51.184118042 +0000 UTC m=+501.679088751" watchObservedRunningTime="2026-09-08 14:06:51.184699432 +0000 UTC m=+501.679670115"
Sep 08 14:06:51.390303 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:51.390274    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:59.390026 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:59.389981    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:06:59.402648 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:06:59.402625    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:07:00.221289 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:00.221257    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-698db7d589-qm6px"
Sep 08 14:07:02.895929 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:02.895891    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"]
Sep 08 14:07:02.900947 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:02.900920    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:02.903969 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:02.903950    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 08 14:07:02.910579 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:02.910547    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"]
Sep 08 14:07:03.002227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.002189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.002227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.002235    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.002456 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.002254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.002456 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.002353    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.002456 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.002400    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.002570 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.002472    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9bgq\" (UniqueName: \"kubernetes.io/projected/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-kube-api-access-p9bgq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.103408 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.103367    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bgq\" (UniqueName: \"kubernetes.io/projected/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-kube-api-access-p9bgq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.103408 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.103415    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.103647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.103451    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.103647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.103471    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.103647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.103491    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.103647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.103530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.103966 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.103944    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.104045 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.103987    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.104045 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.104003    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.106138 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.106115    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.106370 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.106355    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.113296 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.113259    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9bgq\" (UniqueName: \"kubernetes.io/projected/605c5174-b52a-4d85-80f3-1e8fa6bc1f78-kube-api-access-p9bgq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt\" (UID: \"605c5174-b52a-4d85-80f3-1e8fa6bc1f78\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.212933 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.212904    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:03.486790 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:03.486763    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"]
Sep 08 14:07:03.489237 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:07:03.489203    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod605c5174_b52a_4d85_80f3_1e8fa6bc1f78.slice/crio-342bd49ba317b334400a4a52a4ce1b676ccc3dcc50f423995cfcfb815e5c1fa3 WatchSource:0}: Error finding container 342bd49ba317b334400a4a52a4ce1b676ccc3dcc50f423995cfcfb815e5c1fa3: Status 404 returned error can't find the container with id 342bd49ba317b334400a4a52a4ce1b676ccc3dcc50f423995cfcfb815e5c1fa3
Sep 08 14:07:04.232655 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:04.232619    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt" event={"ID":"605c5174-b52a-4d85-80f3-1e8fa6bc1f78","Type":"ContainerStarted","Data":"dc74960cd6292330a2aa3f862fa57be9e3384217eb1f04ad96d36f71c60fab29"}
Sep 08 14:07:04.233068 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:04.232674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt" event={"ID":"605c5174-b52a-4d85-80f3-1e8fa6bc1f78","Type":"ContainerStarted","Data":"342bd49ba317b334400a4a52a4ce1b676ccc3dcc50f423995cfcfb815e5c1fa3"}
Sep 08 14:07:04.255270 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:04.255191    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt" podStartSLOduration=2.054627759 podStartE2EDuration="2.255175189s" totalImagesPullingTime="200.54743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:07:04.254063128 +0000 UTC m=+514.749033812" watchObservedRunningTime="2026-09-08 14:07:04.255175189 +0000 UTC m=+514.750145873"
Sep 08 14:07:05.214035 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:05.214002    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:13.213818 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:13.213786    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:13.226351 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:13.226325    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:07:13.288381 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:07:13.288354    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt"
Sep 08 14:08:33.797793 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:08:33.797676    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:08:33.802565 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:08:33.802547    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:09:20.363284 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.363237    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"]
Sep 08 14:09:20.366589 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.366567    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.369135 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.369114    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 14:09:20.369260 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.369175    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-cb068d-openshift-default-dockercfg-r5jrk\""
Sep 08 14:09:20.379085 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.379059    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"]
Sep 08 14:09:20.439596 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.439565    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-certs\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.439764 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.439604    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-socket\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.439764 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.439623    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-data\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.439764 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.439643    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-credential-socket\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.439764 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.439690    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-envoy\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.439764 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.439741    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istiod-ca-cert\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.440020 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.439782    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-podinfo\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.440020 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.439807    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-token\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.440020 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.439831    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km9xq\" (UniqueName: \"kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-kube-api-access-km9xq\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541083 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541049    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-socket\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541083 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541083    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-data\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541354 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541106    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-credential-socket\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541354 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-envoy\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541354 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541156    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istiod-ca-cert\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541354 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-podinfo\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541354 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541221    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-token\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541354 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541242    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km9xq\" (UniqueName: \"kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-kube-api-access-km9xq\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541354 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-certs\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541713 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541515    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-data\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541713 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541584    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-credential-socket\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541713 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541648    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-socket\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.541713 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541691    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-certs\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.542003 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.541982    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istiod-ca-cert\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.543520 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.543499    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-envoy\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.543969 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.543950    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-podinfo\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.550256 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.550235    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-km9xq\" (UniqueName: \"kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-kube-api-access-km9xq\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.550551 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.550531    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-token\") pod \"e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") " pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.677402 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.677322    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:20.956801 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.956774    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"]
Sep 08 14:09:20.958644 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:09:20.958617    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca21c26b_4510_498d_819f_2bc6fa4a2dae.slice/crio-6ebfd96e0a809ea0977ec3448eafaa273f3bfe64153c86e8c03a6a595f8c292f WatchSource:0}: Error finding container 6ebfd96e0a809ea0977ec3448eafaa273f3bfe64153c86e8c03a6a595f8c292f: Status 404 returned error can't find the container with id 6ebfd96e0a809ea0977ec3448eafaa273f3bfe64153c86e8c03a6a595f8c292f
Sep 08 14:09:20.960761 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:20.960746    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:09:21.876679 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:21.876640    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" event={"ID":"ca21c26b-4510-498d-819f-2bc6fa4a2dae","Type":"ContainerStarted","Data":"6ebfd96e0a809ea0977ec3448eafaa273f3bfe64153c86e8c03a6a595f8c292f"}
Sep 08 14:09:23.626402 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:23.626353    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:09:23.626714 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:23.626449    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:09:23.626714 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:23.626493    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:09:23.889896 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:23.889773    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" event={"ID":"ca21c26b-4510-498d-819f-2bc6fa4a2dae","Type":"ContainerStarted","Data":"2ece0fdd579d060cca9093b0c399fc36ab997de41732a1c215d97dc05eaf2ab0"}
Sep 08 14:09:24.678072 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:24.678034    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:24.684287 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:24.684258    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:24.731246 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:24.731188    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" podStartSLOduration=2.06607067 podStartE2EDuration="4.731172186s" totalImagesPullingTime="2.665101516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:09:23.926365367 +0000 UTC m=+654.421336052" watchObservedRunningTime="2026-09-08 14:09:24.731172186 +0000 UTC m=+655.226142870"
Sep 08 14:09:24.895166 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:24.895138    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:24.896001 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:24.895984    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:35.441094 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.441058    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"]
Sep 08 14:09:35.445008 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.444984    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:09:35.447646 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.447625    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-41383d-dockercfg-6q4kj\""
Sep 08 14:09:35.447774 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.447754    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-41383d\""
Sep 08 14:09:35.460286 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.460264    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"]
Sep 08 14:09:35.577574 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.577537    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4\" (UID: \"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:09:35.577726 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.577662    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v58q9\" (UniqueName: \"kubernetes.io/projected/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-kube-api-access-v58q9\") pod \"payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4\" (UID: \"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:09:35.678588 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.678551    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v58q9\" (UniqueName: \"kubernetes.io/projected/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-kube-api-access-v58q9\") pod \"payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4\" (UID: \"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:09:35.678756 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.678610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4\" (UID: \"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:09:35.679540 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.679521    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4\" (UID: \"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:09:35.692068 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.691999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v58q9\" (UniqueName: \"kubernetes.io/projected/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-kube-api-access-v58q9\") pod \"payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4\" (UID: \"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:09:35.755455 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:35.755422    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:09:36.042785 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.042757    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"]
Sep 08 14:09:36.045304 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:09:36.045270    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ce0c48f_1a10_4edc_8f9c_db8b59cbfa98.slice/crio-38c69ef970c5270502e17a05f72bfac273b09d3b1f25e98f5b932317ea79a0e6 WatchSource:0}: Error finding container 38c69ef970c5270502e17a05f72bfac273b09d3b1f25e98f5b932317ea79a0e6: Status 404 returned error can't find the container with id 38c69ef970c5270502e17a05f72bfac273b09d3b1f25e98f5b932317ea79a0e6
Sep 08 14:09:36.891703 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.891668    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"]
Sep 08 14:09:36.895361 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.895334    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:09:36.898105 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.898081    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-6a4001b4\""
Sep 08 14:09:36.898225 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.898085    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-6a4001b4-dockercfg-wvsjr\""
Sep 08 14:09:36.906687 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.906663    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"]
Sep 08 14:09:36.916909 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.916842    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"]
Sep 08 14:09:36.920528 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.920506    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:09:36.934784 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.934762    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"]
Sep 08 14:09:36.958948 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.958916    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4" event={"ID":"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98","Type":"ContainerStarted","Data":"38c69ef970c5270502e17a05f72bfac273b09d3b1f25e98f5b932317ea79a0e6"}
Sep 08 14:09:36.991641 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.991598    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/196b166d-a2ec-4537-a1d8-2f9ae6816eff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j\" (UID: \"196b166d-a2ec-4537-a1d8-2f9ae6816eff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:09:36.991807 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.991664    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv2kg\" (UniqueName: \"kubernetes.io/projected/196b166d-a2ec-4537-a1d8-2f9ae6816eff-kube-api-access-bv2kg\") pod \"payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j\" (UID: \"196b166d-a2ec-4537-a1d8-2f9ae6816eff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:09:36.991807 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.991689    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77hs4\" (UniqueName: \"kubernetes.io/projected/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-kube-api-access-77hs4\") pod \"payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9\" (UID: \"df018f2a-0b44-4fae-8b8c-87661b5cbe2e\") " pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:09:36.991807 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:36.991758    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-plugins-config-volume\") pod \"payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9\" (UID: \"df018f2a-0b44-4fae-8b8c-87661b5cbe2e\") " pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:09:37.094300 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.093087    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/196b166d-a2ec-4537-a1d8-2f9ae6816eff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j\" (UID: \"196b166d-a2ec-4537-a1d8-2f9ae6816eff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:09:37.094300 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.093160    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv2kg\" (UniqueName: \"kubernetes.io/projected/196b166d-a2ec-4537-a1d8-2f9ae6816eff-kube-api-access-bv2kg\") pod \"payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j\" (UID: \"196b166d-a2ec-4537-a1d8-2f9ae6816eff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:09:37.094300 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.093190    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77hs4\" (UniqueName: \"kubernetes.io/projected/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-kube-api-access-77hs4\") pod \"payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9\" (UID: \"df018f2a-0b44-4fae-8b8c-87661b5cbe2e\") " pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:09:37.094300 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.093255    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-plugins-config-volume\") pod \"payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9\" (UID: \"df018f2a-0b44-4fae-8b8c-87661b5cbe2e\") " pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:09:37.094300 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.094080    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/196b166d-a2ec-4537-a1d8-2f9ae6816eff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j\" (UID: \"196b166d-a2ec-4537-a1d8-2f9ae6816eff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:09:37.094300 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.094247    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-plugins-config-volume\") pod \"payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9\" (UID: \"df018f2a-0b44-4fae-8b8c-87661b5cbe2e\") " pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:09:37.104626 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.104570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77hs4\" (UniqueName: \"kubernetes.io/projected/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-kube-api-access-77hs4\") pod \"payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9\" (UID: \"df018f2a-0b44-4fae-8b8c-87661b5cbe2e\") " pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:09:37.110181 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.110122    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv2kg\" (UniqueName: \"kubernetes.io/projected/196b166d-a2ec-4537-a1d8-2f9ae6816eff-kube-api-access-bv2kg\") pod \"payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j\" (UID: \"196b166d-a2ec-4537-a1d8-2f9ae6816eff\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:09:37.209858 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.209574    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:09:37.235686 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.235656    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:09:37.542426 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.541319    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"]
Sep 08 14:09:37.555476 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:09:37.555436    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod196b166d_a2ec_4537_a1d8_2f9ae6816eff.slice/crio-57e7ac4dab110a9667345a56895d5dd2a7cd3ce8d1457eca32c5783f558afc19 WatchSource:0}: Error finding container 57e7ac4dab110a9667345a56895d5dd2a7cd3ce8d1457eca32c5783f558afc19: Status 404 returned error can't find the container with id 57e7ac4dab110a9667345a56895d5dd2a7cd3ce8d1457eca32c5783f558afc19
Sep 08 14:09:37.561394 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.561344    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"]
Sep 08 14:09:37.563414 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:09:37.563382    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf018f2a_0b44_4fae_8b8c_87661b5cbe2e.slice/crio-9bf2f1c5367da46cd4e19e9889b22aeb2a2d2148ef4181563a08b6fd25033c2e WatchSource:0}: Error finding container 9bf2f1c5367da46cd4e19e9889b22aeb2a2d2148ef4181563a08b6fd25033c2e: Status 404 returned error can't find the container with id 9bf2f1c5367da46cd4e19e9889b22aeb2a2d2148ef4181563a08b6fd25033c2e
Sep 08 14:09:37.964642 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.964608    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j" event={"ID":"196b166d-a2ec-4537-a1d8-2f9ae6816eff","Type":"ContainerStarted","Data":"57e7ac4dab110a9667345a56895d5dd2a7cd3ce8d1457eca32c5783f558afc19"}
Sep 08 14:09:37.966018 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:37.965987    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9" event={"ID":"df018f2a-0b44-4fae-8b8c-87661b5cbe2e","Type":"ContainerStarted","Data":"9bf2f1c5367da46cd4e19e9889b22aeb2a2d2148ef4181563a08b6fd25033c2e"}
Sep 08 14:09:38.973408 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:38.973370    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9" event={"ID":"df018f2a-0b44-4fae-8b8c-87661b5cbe2e","Type":"ContainerStarted","Data":"95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754"}
Sep 08 14:09:38.973999 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:38.973976    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:09:38.975614 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:38.975588    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j" event={"ID":"196b166d-a2ec-4537-a1d8-2f9ae6816eff","Type":"ContainerStarted","Data":"17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096"}
Sep 08 14:09:38.975732 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:38.975668    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:09:38.976845 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:38.976826    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4" event={"ID":"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98","Type":"ContainerStarted","Data":"0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd"}
Sep 08 14:09:38.977085 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:38.977071    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:09:38.991989 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:38.991951    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9" podStartSLOduration=2.2146370810000002 podStartE2EDuration="2.991937006s" totalImagesPullingTime="777.299925ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:09:38.990562517 +0000 UTC m=+669.485533203" watchObservedRunningTime="2026-09-08 14:09:38.991937006 +0000 UTC m=+669.486907690"
Sep 08 14:09:39.009734 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:39.008963    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j" podStartSLOduration=2.23143382 podStartE2EDuration="3.008949874s" totalImagesPullingTime="777.516054ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:09:39.007816034 +0000 UTC m=+669.502786718" watchObservedRunningTime="2026-09-08 14:09:39.008949874 +0000 UTC m=+669.503920558"
Sep 08 14:09:39.030552 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:39.028919    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4" podStartSLOduration=1.7390425 podStartE2EDuration="4.028898504s" totalImagesPullingTime="2.289856004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:09:39.025802124 +0000 UTC m=+669.520772813" watchObservedRunningTime="2026-09-08 14:09:39.028898504 +0000 UTC m=+669.523869166"
Sep 08 14:09:47.022713 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:47.022680    2564 generic.go:356] "Generic (PLEG): container finished" podID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerID="2ece0fdd579d060cca9093b0c399fc36ab997de41732a1c215d97dc05eaf2ab0" exitCode=0
Sep 08 14:09:47.023276 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:47.022760    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" event={"ID":"ca21c26b-4510-498d-819f-2bc6fa4a2dae","Type":"ContainerDied","Data":"2ece0fdd579d060cca9093b0c399fc36ab997de41732a1c215d97dc05eaf2ab0"}
Sep 08 14:09:47.023350 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:47.023298    2564 scope.go:117] "RemoveContainer" containerID="2ece0fdd579d060cca9093b0c399fc36ab997de41732a1c215d97dc05eaf2ab0"
Sep 08 14:09:47.024529 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:47.024500    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:09:47.024611 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:47.024569    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:09:47.024611 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:47.024603    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:09:48.030510 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:48.030471    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" event={"ID":"ca21c26b-4510-498d-819f-2bc6fa4a2dae","Type":"ContainerStarted","Data":"e91b2138d980ff667a7252148d1734d5237864ded03c880e1a3168a595c67c7a"}
Sep 08 14:09:48.677793 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:48.677752    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:48.678072 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:48.678045    2564 patch_prober.go:28] interesting pod/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 08 14:09:48.678139 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:48.678114    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 08 14:09:49.678069 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:49.678032    2564 patch_prober.go:28] interesting pod/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 08 14:09:49.678526 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:49.678105    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 08 14:09:49.984047 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:49.984014    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:09:49.984259 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:49.984194    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:09:50.677909 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:50.677847    2564 patch_prober.go:28] interesting pod/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 08 14:09:50.678111 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:50.677934    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 08 14:09:50.989785 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:50.989755    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:09:51.539890 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.536596    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"]
Sep 08 14:09:51.552905 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.552256    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.555233 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.555208    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-94eadd-openshift-default-dockercfg-qh57x\""
Sep 08 14:09:51.557427 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.557401    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"]
Sep 08 14:09:51.655725 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.655680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-socket\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.655947 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.655750    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-certs\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.655947 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.655766    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-envoy\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.655947 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.655785    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6rkh\" (UniqueName: \"kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-kube-api-access-v6rkh\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.655947 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.655809    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-token\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.655947 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.655828    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-credential-socket\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.655947 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.655848    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-podinfo\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.655947 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.655883    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istiod-ca-cert\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.655947 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.655909    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-data\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.677720 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.677687    2564 patch_prober.go:28] interesting pod/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 08 14:09:51.677850 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.677740    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 08 14:09:51.757384 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.757347    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6rkh\" (UniqueName: \"kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-kube-api-access-v6rkh\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.757837 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.757403    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-token\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.757837 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.757434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-credential-socket\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.757837 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.757466    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-podinfo\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.757837 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.757498    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istiod-ca-cert\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.757837 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.757530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-data\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.757837 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.757702    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-socket\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.757837 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.757816    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-certs\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.758270 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.757844    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-envoy\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.758270 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.758077    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-credential-socket\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.758378 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.758263    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-certs\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.758439 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.758414    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-data\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.758488 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.758445    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-socket\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.758488 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.758449    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istiod-ca-cert\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.760619 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.760601    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-envoy\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.760878 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.760847    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-podinfo\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.766282 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.766264    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-token\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.766546 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.766525    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6rkh\" (UniqueName: \"kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-kube-api-access-v6rkh\") pod \"e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") " pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:51.866509 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:51.866427    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:52.161197 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:52.161171    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"]
Sep 08 14:09:52.162968 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:09:52.162938    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ed6dea7_cccc_4bfd_a9ca_72346ae7d247.slice/crio-41219db537975790c9ac51caf663c6eeaff97148844756efb52945b25332c7a1 WatchSource:0}: Error finding container 41219db537975790c9ac51caf663c6eeaff97148844756efb52945b25332c7a1: Status 404 returned error can't find the container with id 41219db537975790c9ac51caf663c6eeaff97148844756efb52945b25332c7a1
Sep 08 14:09:52.165502 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:52.165455    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:09:52.165720 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:52.165690    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:09:52.165820 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:52.165761    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:09:52.678728 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:52.678697    2564 patch_prober.go:28] interesting pod/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 08 14:09:52.678938 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:52.678751    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 08 14:09:53.072639 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:53.072600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" event={"ID":"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247","Type":"ContainerStarted","Data":"e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743"}
Sep 08 14:09:53.073103 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:53.072657    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" event={"ID":"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247","Type":"ContainerStarted","Data":"41219db537975790c9ac51caf663c6eeaff97148844756efb52945b25332c7a1"}
Sep 08 14:09:53.095411 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:53.095362    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" podStartSLOduration=2.0953459309999998 podStartE2EDuration="2.095345931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:09:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:09:53.093277286 +0000 UTC m=+683.588247971" watchObservedRunningTime="2026-09-08 14:09:53.095345931 +0000 UTC m=+683.590316615"
Sep 08 14:09:53.678446 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:53.678406    2564 patch_prober.go:28] interesting pod/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 08 14:09:53.678622 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:53.678478    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 08 14:09:53.866855 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:53.866810    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:53.873570 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:53.873545    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:54.079138 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:54.079094    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:54.080438 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:54.080409    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:09:54.682019 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:54.681991    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:54.682404 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:54.682382    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:54.683039 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:54.683023    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:09:57.204128 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.204086    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"]
Sep 08 14:09:57.208244 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.208176    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:09:57.210831 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.210800    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-94eadd\""
Sep 08 14:09:57.210995 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.210950    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-94eadd-dockercfg-xzgl7\""
Sep 08 14:09:57.223212 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.223182    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"]
Sep 08 14:09:57.316270 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.316239    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5j2q\" (UniqueName: \"kubernetes.io/projected/77b9b8ff-c7b4-482a-953f-a8211f69e50c-kube-api-access-n5j2q\") pod \"payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj\" (UID: \"77b9b8ff-c7b4-482a-953f-a8211f69e50c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:09:57.316432 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.316284    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77b9b8ff-c7b4-482a-953f-a8211f69e50c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj\" (UID: \"77b9b8ff-c7b4-482a-953f-a8211f69e50c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:09:57.417316 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.417275    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5j2q\" (UniqueName: \"kubernetes.io/projected/77b9b8ff-c7b4-482a-953f-a8211f69e50c-kube-api-access-n5j2q\") pod \"payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj\" (UID: \"77b9b8ff-c7b4-482a-953f-a8211f69e50c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:09:57.417480 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.417345    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77b9b8ff-c7b4-482a-953f-a8211f69e50c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj\" (UID: \"77b9b8ff-c7b4-482a-953f-a8211f69e50c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:09:57.418050 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.418032    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77b9b8ff-c7b4-482a-953f-a8211f69e50c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj\" (UID: \"77b9b8ff-c7b4-482a-953f-a8211f69e50c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:09:57.426716 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.426682    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5j2q\" (UniqueName: \"kubernetes.io/projected/77b9b8ff-c7b4-482a-953f-a8211f69e50c-kube-api-access-n5j2q\") pod \"payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj\" (UID: \"77b9b8ff-c7b4-482a-953f-a8211f69e50c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:09:57.518976 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.518942    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:09:57.796802 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:57.796728    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"]
Sep 08 14:09:57.800379 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:09:57.800346    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b9b8ff_c7b4_482a_953f_a8211f69e50c.slice/crio-4defcad23a24eaa6104e07dd28713a399d2375a7fbac51e601fbfccddaa26046 WatchSource:0}: Error finding container 4defcad23a24eaa6104e07dd28713a399d2375a7fbac51e601fbfccddaa26046: Status 404 returned error can't find the container with id 4defcad23a24eaa6104e07dd28713a399d2375a7fbac51e601fbfccddaa26046
Sep 08 14:09:58.102134 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:58.102047    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj" event={"ID":"77b9b8ff-c7b4-482a-953f-a8211f69e50c","Type":"ContainerStarted","Data":"59757cf7d6da6299fbe6eb5c1affea78954ba7120058860ed46d25cbdff8caca"}
Sep 08 14:09:58.102134 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:58.102105    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj" event={"ID":"77b9b8ff-c7b4-482a-953f-a8211f69e50c","Type":"ContainerStarted","Data":"4defcad23a24eaa6104e07dd28713a399d2375a7fbac51e601fbfccddaa26046"}
Sep 08 14:09:58.102134 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:58.102136    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:09:58.118777 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:09:58.118668    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj" podStartSLOduration=1.118648736 podStartE2EDuration="1.118648736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:09:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:09:58.117833415 +0000 UTC m=+688.612804100" watchObservedRunningTime="2026-09-08 14:09:58.118648736 +0000 UTC m=+688.613619421"
Sep 08 14:10:04.472212 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:04.472176    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"]
Sep 08 14:10:04.475920 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:04.475892    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"
Sep 08 14:10:04.478351 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:04.478329    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:10:04.478351 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:04.478342    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:10:04.482459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:04.482435    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"]
Sep 08 14:10:04.591973 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:04.591931    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh\" (UID: \"cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"
Sep 08 14:10:04.692607 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:04.692573    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh\" (UID: \"cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"
Sep 08 14:10:04.693215 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:04.693195    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh\" (UID: \"cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"
Sep 08 14:10:04.789110 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:04.789016    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"
Sep 08 14:10:05.065124 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:05.065102    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"]
Sep 08 14:10:05.067336 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:10:05.067296    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb5c9c00_52ec_4e72_a9a6_3ede5c0968fe.slice/crio-c6d7ef908da0e165b29042cfefc6ee27de9e45e1d07f6dca21e3ac0129790daa WatchSource:0}: Error finding container c6d7ef908da0e165b29042cfefc6ee27de9e45e1d07f6dca21e3ac0129790daa: Status 404 returned error can't find the container with id c6d7ef908da0e165b29042cfefc6ee27de9e45e1d07f6dca21e3ac0129790daa
Sep 08 14:10:05.140764 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:05.140725    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh" event={"ID":"cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe","Type":"ContainerStarted","Data":"c6d7ef908da0e165b29042cfefc6ee27de9e45e1d07f6dca21e3ac0129790daa"}
Sep 08 14:10:09.109300 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:09.109275    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:10:10.169747 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:10.169711    2564 generic.go:356] "Generic (PLEG): container finished" podID="cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe" containerID="98f0fb267a1d2c6224272354a834870bd1c2673017b0e98d27733f089037a552" exitCode=0
Sep 08 14:10:10.170252 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:10.169805    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh" event={"ID":"cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe","Type":"ContainerDied","Data":"98f0fb267a1d2c6224272354a834870bd1c2673017b0e98d27733f089037a552"}
Sep 08 14:10:11.359270 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:11.359242    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"
Sep 08 14:10:11.464299 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:11.464272    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe-maas-api-service-ca\") pod \"cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe\" (UID: \"cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe\") "
Sep 08 14:10:11.464645 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:11.464621    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe" (UID: "cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:10:11.565855 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:11.565820    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:12.180706 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:12.180678    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"
Sep 08 14:10:12.180706 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:12.180687    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh" event={"ID":"cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe","Type":"ContainerDied","Data":"c6d7ef908da0e165b29042cfefc6ee27de9e45e1d07f6dca21e3ac0129790daa"}
Sep 08 14:10:12.180964 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:12.180740    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6d7ef908da0e165b29042cfefc6ee27de9e45e1d07f6dca21e3ac0129790daa"
Sep 08 14:10:15.539677 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.539637    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"]
Sep 08 14:10:15.540302 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.539906    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9" podUID="df018f2a-0b44-4fae-8b8c-87661b5cbe2e" containerName="payload-processing" containerID="cri-o://95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754" gracePeriod=30
Sep 08 14:10:15.550253 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.550191    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"]
Sep 08 14:10:15.550478 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.550415    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j" podUID="196b166d-a2ec-4537-a1d8-2f9ae6816eff" containerName="payload-pre-processing" containerID="cri-o://17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096" gracePeriod=30
Sep 08 14:10:15.752046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.751998    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"]
Sep 08 14:10:15.756551 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.756521    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6a4001b4-rn5bh"]
Sep 08 14:10:15.813274 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.813247    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:10:15.854569 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.854542    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:10:15.910350 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.910313    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77hs4\" (UniqueName: \"kubernetes.io/projected/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-kube-api-access-77hs4\") pod \"df018f2a-0b44-4fae-8b8c-87661b5cbe2e\" (UID: \"df018f2a-0b44-4fae-8b8c-87661b5cbe2e\") "
Sep 08 14:10:15.910528 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.910376    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-plugins-config-volume\") pod \"df018f2a-0b44-4fae-8b8c-87661b5cbe2e\" (UID: \"df018f2a-0b44-4fae-8b8c-87661b5cbe2e\") "
Sep 08 14:10:15.910940 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.910854    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df018f2a-0b44-4fae-8b8c-87661b5cbe2e" (UID: "df018f2a-0b44-4fae-8b8c-87661b5cbe2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:10:15.913172 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.913146    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-kube-api-access-77hs4" (OuterVolumeSpecName: "kube-api-access-77hs4") pod "df018f2a-0b44-4fae-8b8c-87661b5cbe2e" (UID: "df018f2a-0b44-4fae-8b8c-87661b5cbe2e"). InnerVolumeSpecName "kube-api-access-77hs4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:10:15.961481 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:15.961432    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe" path="/var/lib/kubelet/pods/cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe/volumes"
Sep 08 14:10:16.011610 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.011572    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/196b166d-a2ec-4537-a1d8-2f9ae6816eff-plugins-config-volume\") pod \"196b166d-a2ec-4537-a1d8-2f9ae6816eff\" (UID: \"196b166d-a2ec-4537-a1d8-2f9ae6816eff\") "
Sep 08 14:10:16.011801 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.011643    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv2kg\" (UniqueName: \"kubernetes.io/projected/196b166d-a2ec-4537-a1d8-2f9ae6816eff-kube-api-access-bv2kg\") pod \"196b166d-a2ec-4537-a1d8-2f9ae6816eff\" (UID: \"196b166d-a2ec-4537-a1d8-2f9ae6816eff\") "
Sep 08 14:10:16.012097 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.011940    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77hs4\" (UniqueName: \"kubernetes.io/projected/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-kube-api-access-77hs4\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:16.012097 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.011967    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df018f2a-0b44-4fae-8b8c-87661b5cbe2e-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:16.012232 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.012137    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/196b166d-a2ec-4537-a1d8-2f9ae6816eff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "196b166d-a2ec-4537-a1d8-2f9ae6816eff" (UID: "196b166d-a2ec-4537-a1d8-2f9ae6816eff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:10:16.014119 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.014094    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/196b166d-a2ec-4537-a1d8-2f9ae6816eff-kube-api-access-bv2kg" (OuterVolumeSpecName: "kube-api-access-bv2kg") pod "196b166d-a2ec-4537-a1d8-2f9ae6816eff" (UID: "196b166d-a2ec-4537-a1d8-2f9ae6816eff"). InnerVolumeSpecName "kube-api-access-bv2kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:10:16.112902 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.112783    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/196b166d-a2ec-4537-a1d8-2f9ae6816eff-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:16.112902 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.112816    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv2kg\" (UniqueName: \"kubernetes.io/projected/196b166d-a2ec-4537-a1d8-2f9ae6816eff-kube-api-access-bv2kg\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:16.202878 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.202834    2564 generic.go:356] "Generic (PLEG): container finished" podID="196b166d-a2ec-4537-a1d8-2f9ae6816eff" containerID="17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096" exitCode=0
Sep 08 14:10:16.203046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.202894    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"
Sep 08 14:10:16.203046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.202934    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j" event={"ID":"196b166d-a2ec-4537-a1d8-2f9ae6816eff","Type":"ContainerDied","Data":"17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096"}
Sep 08 14:10:16.203046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.203001    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j" event={"ID":"196b166d-a2ec-4537-a1d8-2f9ae6816eff","Type":"ContainerDied","Data":"57e7ac4dab110a9667345a56895d5dd2a7cd3ce8d1457eca32c5783f558afc19"}
Sep 08 14:10:16.203046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.203019    2564 scope.go:117] "RemoveContainer" containerID="17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096"
Sep 08 14:10:16.204594 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.204564    2564 generic.go:356] "Generic (PLEG): container finished" podID="df018f2a-0b44-4fae-8b8c-87661b5cbe2e" containerID="95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754" exitCode=0
Sep 08 14:10:16.204723 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.204594    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9" event={"ID":"df018f2a-0b44-4fae-8b8c-87661b5cbe2e","Type":"ContainerDied","Data":"95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754"}
Sep 08 14:10:16.204723 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.204639    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9" event={"ID":"df018f2a-0b44-4fae-8b8c-87661b5cbe2e","Type":"ContainerDied","Data":"9bf2f1c5367da46cd4e19e9889b22aeb2a2d2148ef4181563a08b6fd25033c2e"}
Sep 08 14:10:16.204835 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.204791    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"
Sep 08 14:10:16.213675 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.213649    2564 scope.go:117] "RemoveContainer" containerID="17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096"
Sep 08 14:10:16.214061 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:10:16.214036    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096\": container with ID starting with 17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096 not found: ID does not exist" containerID="17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096"
Sep 08 14:10:16.214176 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.214067    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096"} err="failed to get container status \"17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096\": rpc error: code = NotFound desc = could not find container \"17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096\": container with ID starting with 17eb68be0c1bd01dcb94bec5541b676915a7507cd6b54bfaaa65ff36f526c096 not found: ID does not exist"
Sep 08 14:10:16.214176 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.214084    2564 scope.go:117] "RemoveContainer" containerID="95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754"
Sep 08 14:10:16.224457 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.224403    2564 scope.go:117] "RemoveContainer" containerID="95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754"
Sep 08 14:10:16.224824 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:10:16.224797    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754\": container with ID starting with 95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754 not found: ID does not exist" containerID="95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754"
Sep 08 14:10:16.224980 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.224832    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754"} err="failed to get container status \"95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754\": rpc error: code = NotFound desc = could not find container \"95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754\": container with ID starting with 95654dc7d7690aa14dff401e0adeb53195ea42e386600dd37c989bba4a12c754 not found: ID does not exist"
Sep 08 14:10:16.226806 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.226785    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"]
Sep 08 14:10:16.233779 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.233754    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-6a4001b4-66898dcddc-rkgs9"]
Sep 08 14:10:16.243385 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.243359    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"]
Sep 08 14:10:16.247383 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.247358    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-6a4001b4-5f584cc854-6h78j"]
Sep 08 14:10:16.821544 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.821462    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"]
Sep 08 14:10:16.822043 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.822025    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="196b166d-a2ec-4537-a1d8-2f9ae6816eff" containerName="payload-pre-processing"
Sep 08 14:10:16.822105 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.822045    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="196b166d-a2ec-4537-a1d8-2f9ae6816eff" containerName="payload-pre-processing"
Sep 08 14:10:16.822105 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.822061    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe" containerName="revoke-keys"
Sep 08 14:10:16.822105 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.822070    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe" containerName="revoke-keys"
Sep 08 14:10:16.822254 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.822107    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df018f2a-0b44-4fae-8b8c-87661b5cbe2e" containerName="payload-processing"
Sep 08 14:10:16.822254 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.822116    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="df018f2a-0b44-4fae-8b8c-87661b5cbe2e" containerName="payload-processing"
Sep 08 14:10:16.822254 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.822198    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="196b166d-a2ec-4537-a1d8-2f9ae6816eff" containerName="payload-pre-processing"
Sep 08 14:10:16.822254 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.822212    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="df018f2a-0b44-4fae-8b8c-87661b5cbe2e" containerName="payload-processing"
Sep 08 14:10:16.822254 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.822222    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb5c9c00-52ec-4e72-a9a6-3ede5c0968fe" containerName="revoke-keys"
Sep 08 14:10:16.828411 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.828385    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:16.831277 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.831252    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-f6640f0d-gw-openshift-default-dockercfg-p6fl9\""
Sep 08 14:10:16.842831 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.842806    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"]
Sep 08 14:10:16.920176 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.920132    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-envoy\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:16.920363 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.920180    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-podinfo\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:16.920363 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.920280    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-socket\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:16.920363 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.920328    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-data\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:16.920485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.920369    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-certs\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:16.920485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.920394    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ca42b31-9962-4880-98d9-67f712cd5c5f-istiod-ca-cert\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:16.920485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.920439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-credential-socket\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:16.920485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.920467    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-token\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:16.920629 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:16.920493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crqm6\" (UniqueName: \"kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-kube-api-access-crqm6\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.021573 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.021537    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-certs\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.021752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.021577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ca42b31-9962-4880-98d9-67f712cd5c5f-istiod-ca-cert\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.021752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.021617    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-credential-socket\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.021752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.021653    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-token\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.021752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.021687    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crqm6\" (UniqueName: \"kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-kube-api-access-crqm6\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.021752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.021750    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-envoy\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.022046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.021781    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-podinfo\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.022046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.021842    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-socket\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.022046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.021923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-data\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.022046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.022028    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-certs\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.022245 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.022129    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-credential-socket\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.022301 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.022275    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-data\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.022384 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.022355    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ca42b31-9962-4880-98d9-67f712cd5c5f-istiod-ca-cert\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.022384 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.022380    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-socket\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.024480 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.024452    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-envoy\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.024928 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.024904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-podinfo\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.047136 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.047105    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-token\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.047136 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.047125    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crqm6\" (UniqueName: \"kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-kube-api-access-crqm6\") pod \"e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") " pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.139119 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.139024    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:17.442391 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.442360    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"]
Sep 08 14:10:17.444423 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:10:17.444396    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca42b31_9962_4880_98d9_67f712cd5c5f.slice/crio-e5bd5b3e048729361f0e20366e7ea58077e34212f79563162d6c376cc730a356 WatchSource:0}: Error finding container e5bd5b3e048729361f0e20366e7ea58077e34212f79563162d6c376cc730a356: Status 404 returned error can't find the container with id e5bd5b3e048729361f0e20366e7ea58077e34212f79563162d6c376cc730a356
Sep 08 14:10:17.447115 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.447059    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:10:17.447217 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.447182    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:10:17.447260 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.447231    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:10:17.960752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.960717    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="196b166d-a2ec-4537-a1d8-2f9ae6816eff" path="/var/lib/kubelet/pods/196b166d-a2ec-4537-a1d8-2f9ae6816eff/volumes"
Sep 08 14:10:17.961242 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:17.961126    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df018f2a-0b44-4fae-8b8c-87661b5cbe2e" path="/var/lib/kubelet/pods/df018f2a-0b44-4fae-8b8c-87661b5cbe2e/volumes"
Sep 08 14:10:18.223662 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:18.223629    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" event={"ID":"1ca42b31-9962-4880-98d9-67f712cd5c5f","Type":"ContainerStarted","Data":"24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd"}
Sep 08 14:10:18.223852 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:18.223699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" event={"ID":"1ca42b31-9962-4880-98d9-67f712cd5c5f","Type":"ContainerStarted","Data":"e5bd5b3e048729361f0e20366e7ea58077e34212f79563162d6c376cc730a356"}
Sep 08 14:10:18.247626 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:18.247535    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" podStartSLOduration=2.247516258 podStartE2EDuration="2.247516258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:10:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:10:18.246529383 +0000 UTC m=+708.741500079" watchObservedRunningTime="2026-09-08 14:10:18.247516258 +0000 UTC m=+708.742486943"
Sep 08 14:10:19.139632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:19.139594    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:19.144888 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:19.144832    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:19.231375 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:19.231343    2564 generic.go:356] "Generic (PLEG): container finished" podID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerID="e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743" exitCode=0
Sep 08 14:10:19.231569 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:19.231421    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" event={"ID":"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247","Type":"ContainerDied","Data":"e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743"}
Sep 08 14:10:19.231698 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:19.231673    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:19.232063 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:19.232044    2564 scope.go:117] "RemoveContainer" containerID="e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743"
Sep 08 14:10:19.232908 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:19.232699    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:19.233206 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:19.233176    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:10:19.233263 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:19.233249    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:10:19.233325 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:19.233312    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:10:20.244513 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.244469    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" event={"ID":"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247","Type":"ContainerStarted","Data":"3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc"}
Sep 08 14:10:20.700490 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.700416    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"]
Sep 08 14:10:20.702888 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.702845    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:20.705545 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.705521    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-f6640f0d-dockercfg-dxdq7\""
Sep 08 14:10:20.705652 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.705521    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-f6640f0d\""
Sep 08 14:10:20.710150 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.710094    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"]
Sep 08 14:10:20.712448 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.712430    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:20.720278 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.720257    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"]
Sep 08 14:10:20.735888 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.733277    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"]
Sep 08 14:10:20.862727 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.862683    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82ca4424-326e-4fd7-8a64-7d8712aa8b07-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl\" (UID: \"82ca4424-326e-4fd7-8a64-7d8712aa8b07\") " pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:20.862921 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.862898    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x9lm\" (UniqueName: \"kubernetes.io/projected/98075290-621f-4477-8123-4017c06f943b-kube-api-access-2x9lm\") pod \"payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq\" (UID: \"98075290-621f-4477-8123-4017c06f943b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:20.862980 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.862952    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98075290-621f-4477-8123-4017c06f943b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq\" (UID: \"98075290-621f-4477-8123-4017c06f943b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:20.863033 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.862998    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmgvc\" (UniqueName: \"kubernetes.io/projected/82ca4424-326e-4fd7-8a64-7d8712aa8b07-kube-api-access-kmgvc\") pod \"payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl\" (UID: \"82ca4424-326e-4fd7-8a64-7d8712aa8b07\") " pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:20.866723 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.866577    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:10:20.867768 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.867743    2564 patch_prober.go:28] interesting pod/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 08 14:10:20.867901 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.867802    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 08 14:10:20.964067 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.964036    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmgvc\" (UniqueName: \"kubernetes.io/projected/82ca4424-326e-4fd7-8a64-7d8712aa8b07-kube-api-access-kmgvc\") pod \"payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl\" (UID: \"82ca4424-326e-4fd7-8a64-7d8712aa8b07\") " pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:20.964300 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.964280    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82ca4424-326e-4fd7-8a64-7d8712aa8b07-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl\" (UID: \"82ca4424-326e-4fd7-8a64-7d8712aa8b07\") " pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:20.964567 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.964548    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x9lm\" (UniqueName: \"kubernetes.io/projected/98075290-621f-4477-8123-4017c06f943b-kube-api-access-2x9lm\") pod \"payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq\" (UID: \"98075290-621f-4477-8123-4017c06f943b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:20.964725 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.964708    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98075290-621f-4477-8123-4017c06f943b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq\" (UID: \"98075290-621f-4477-8123-4017c06f943b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:20.965136 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.965112    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82ca4424-326e-4fd7-8a64-7d8712aa8b07-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl\" (UID: \"82ca4424-326e-4fd7-8a64-7d8712aa8b07\") " pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:20.965566 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.965542    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98075290-621f-4477-8123-4017c06f943b-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq\" (UID: \"98075290-621f-4477-8123-4017c06f943b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:20.990508 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:20.990474    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmgvc\" (UniqueName: \"kubernetes.io/projected/82ca4424-326e-4fd7-8a64-7d8712aa8b07-kube-api-access-kmgvc\") pod \"payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl\" (UID: \"82ca4424-326e-4fd7-8a64-7d8712aa8b07\") " pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:21.001372 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:21.001346    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x9lm\" (UniqueName: \"kubernetes.io/projected/98075290-621f-4477-8123-4017c06f943b-kube-api-access-2x9lm\") pod \"payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq\" (UID: \"98075290-621f-4477-8123-4017c06f943b\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:21.015457 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:21.015431    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:21.051843 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:21.051049    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:21.340955 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:21.337420    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"]
Sep 08 14:10:21.391730 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:21.391695    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"]
Sep 08 14:10:21.395637 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:10:21.395612    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ca4424_326e_4fd7_8a64_7d8712aa8b07.slice/crio-6eafbdc1acb9ee304bcbdbc7a9181db1ae0e4d853093c240def599a641a33f90 WatchSource:0}: Error finding container 6eafbdc1acb9ee304bcbdbc7a9181db1ae0e4d853093c240def599a641a33f90: Status 404 returned error can't find the container with id 6eafbdc1acb9ee304bcbdbc7a9181db1ae0e4d853093c240def599a641a33f90
Sep 08 14:10:21.867766 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:21.867732    2564 patch_prober.go:28] interesting pod/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 08 14:10:21.868061 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:21.868032    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 08 14:10:22.270621 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:22.270554    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl" event={"ID":"82ca4424-326e-4fd7-8a64-7d8712aa8b07","Type":"ContainerStarted","Data":"568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad"}
Sep 08 14:10:22.270961 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:22.270658    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:22.270961 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:22.270688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl" event={"ID":"82ca4424-326e-4fd7-8a64-7d8712aa8b07","Type":"ContainerStarted","Data":"6eafbdc1acb9ee304bcbdbc7a9181db1ae0e4d853093c240def599a641a33f90"}
Sep 08 14:10:22.272655 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:22.272629    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq" event={"ID":"98075290-621f-4477-8123-4017c06f943b","Type":"ContainerStarted","Data":"eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf"}
Sep 08 14:10:22.272778 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:22.272679    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq" event={"ID":"98075290-621f-4477-8123-4017c06f943b","Type":"ContainerStarted","Data":"63cbf58041fc3ef0bbbfd6014a8dc22b20020287004e39b7a5e25c42c301ecb7"}
Sep 08 14:10:22.272836 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:22.272774    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:22.313962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:22.313904    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq" podStartSLOduration=2.313882761 podStartE2EDuration="2.313882761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:10:22.31240398 +0000 UTC m=+712.807374664" watchObservedRunningTime="2026-09-08 14:10:22.313882761 +0000 UTC m=+712.808853448"
Sep 08 14:10:22.315360 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:22.315322    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl" podStartSLOduration=2.315310862 podStartE2EDuration="2.315310862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:10:22.291323854 +0000 UTC m=+712.786294538" watchObservedRunningTime="2026-09-08 14:10:22.315310862 +0000 UTC m=+712.810281546"
Sep 08 14:10:22.867581 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:22.867536    2564 patch_prober.go:28] interesting pod/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 08 14:10:22.868097 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:22.867606    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 08 14:10:23.867820 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:23.867775    2564 patch_prober.go:28] interesting pod/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 08 14:10:23.868258 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:23.867834    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 08 14:10:24.079327 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:24.079289    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:10:24.867419 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:24.867382    2564 patch_prober.go:28] interesting pod/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Sep 08 14:10:24.867606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:24.867438    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Sep 08 14:10:25.900302 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:25.900274    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:10:25.901692 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:25.901665    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:10:33.280695 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:33.280660    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:33.281195 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:33.280747    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:38.371646 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:38.371611    2564 generic.go:356] "Generic (PLEG): container finished" podID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerID="24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd" exitCode=0
Sep 08 14:10:38.372121 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:38.371699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" event={"ID":"1ca42b31-9962-4880-98d9-67f712cd5c5f","Type":"ContainerDied","Data":"24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd"}
Sep 08 14:10:38.372237 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:38.372223    2564 scope.go:117] "RemoveContainer" containerID="24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd"
Sep 08 14:10:38.373427 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:38.373397    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:10:38.373489 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:38.373464    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:10:38.373524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:38.373497    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:10:39.379225 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:39.379191    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" event={"ID":"1ca42b31-9962-4880-98d9-67f712cd5c5f","Type":"ContainerStarted","Data":"bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398"}
Sep 08 14:10:40.139442 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:40.139408    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:40.139652 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:40.139525    2564 patch_prober.go:28] interesting pod/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 08 14:10:40.139652 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:40.139568    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 08 14:10:41.139884 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:41.139820    2564 patch_prober.go:28] interesting pod/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 08 14:10:41.140380 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:41.139911    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 08 14:10:42.140372 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:42.140325    2564 patch_prober.go:28] interesting pod/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 08 14:10:42.140891 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:42.140408    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 08 14:10:43.139854 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:43.139815    2564 patch_prober.go:28] interesting pod/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 08 14:10:43.140201 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:43.140169    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 08 14:10:44.140446 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:44.140398    2564 patch_prober.go:28] interesting pod/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 08 14:10:44.141117 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:44.141086    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 08 14:10:44.711606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:44.711571    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"]
Sep 08 14:10:44.714541 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:44.714519    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"
Sep 08 14:10:44.717095 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:44.717071    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:10:44.717272 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:44.717257    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:10:44.724221 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:44.724199    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"]
Sep 08 14:10:44.808909 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:44.808837    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7259b36e-f673-431b-ba1b-05131cd625f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5\" (UID: \"7259b36e-f673-431b-ba1b-05131cd625f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"
Sep 08 14:10:44.910541 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:44.910498    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7259b36e-f673-431b-ba1b-05131cd625f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5\" (UID: \"7259b36e-f673-431b-ba1b-05131cd625f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"
Sep 08 14:10:44.911155 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:44.911137    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7259b36e-f673-431b-ba1b-05131cd625f3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5\" (UID: \"7259b36e-f673-431b-ba1b-05131cd625f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"
Sep 08 14:10:45.026625 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:45.026588    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"
Sep 08 14:10:45.141514 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:45.140421    2564 patch_prober.go:28] interesting pod/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 08 14:10:45.141514 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:45.140496    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 08 14:10:45.316249 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:45.316226    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"]
Sep 08 14:10:45.317966 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:10:45.317928    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7259b36e_f673_431b_ba1b_05131cd625f3.slice/crio-41ecb198aa1cb48f1132d928639eb4763852a3371628badc18312e8c450153a4 WatchSource:0}: Error finding container 41ecb198aa1cb48f1132d928639eb4763852a3371628badc18312e8c450153a4: Status 404 returned error can't find the container with id 41ecb198aa1cb48f1132d928639eb4763852a3371628badc18312e8c450153a4
Sep 08 14:10:45.412137 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:45.412102    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5" event={"ID":"7259b36e-f673-431b-ba1b-05131cd625f3","Type":"ContainerStarted","Data":"41ecb198aa1cb48f1132d928639eb4763852a3371628badc18312e8c450153a4"}
Sep 08 14:10:46.144180 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:46.144155    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:46.144611 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:46.144588    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:46.145292 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:46.145270    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:46.419168 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:46.419067    2564 generic.go:356] "Generic (PLEG): container finished" podID="7259b36e-f673-431b-ba1b-05131cd625f3" containerID="a3bf6df12709c932644572ae9d0f31fa20ab3bd805004c057c66bdcecc2721f2" exitCode=0
Sep 08 14:10:46.419323 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:46.419161    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5" event={"ID":"7259b36e-f673-431b-ba1b-05131cd625f3","Type":"ContainerDied","Data":"a3bf6df12709c932644572ae9d0f31fa20ab3bd805004c057c66bdcecc2721f2"}
Sep 08 14:10:47.562227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:47.562207    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"
Sep 08 14:10:47.641589 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:47.641554    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7259b36e-f673-431b-ba1b-05131cd625f3-maas-api-service-ca\") pod \"7259b36e-f673-431b-ba1b-05131cd625f3\" (UID: \"7259b36e-f673-431b-ba1b-05131cd625f3\") "
Sep 08 14:10:47.641914 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:47.641891    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7259b36e-f673-431b-ba1b-05131cd625f3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7259b36e-f673-431b-ba1b-05131cd625f3" (UID: "7259b36e-f673-431b-ba1b-05131cd625f3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:10:47.642060 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:47.642044    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7259b36e-f673-431b-ba1b-05131cd625f3-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:48.430618 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:48.430523    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5" event={"ID":"7259b36e-f673-431b-ba1b-05131cd625f3","Type":"ContainerDied","Data":"41ecb198aa1cb48f1132d928639eb4763852a3371628badc18312e8c450153a4"}
Sep 08 14:10:48.430618 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:48.430601    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41ecb198aa1cb48f1132d928639eb4763852a3371628badc18312e8c450153a4"
Sep 08 14:10:48.430618 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:48.430546    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"
Sep 08 14:10:50.646329 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:50.646264    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"]
Sep 08 14:10:50.646761 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:50.646598    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl" podUID="82ca4424-326e-4fd7-8a64-7d8712aa8b07" containerName="payload-processing" containerID="cri-o://568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad" gracePeriod=30
Sep 08 14:10:50.655642 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:50.655600    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"]
Sep 08 14:10:50.656143 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:50.656045    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq" podUID="98075290-621f-4477-8123-4017c06f943b" containerName="payload-pre-processing" containerID="cri-o://eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf" gracePeriod=30
Sep 08 14:10:50.830417 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:50.830387    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"]
Sep 08 14:10:50.831985 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:50.831962    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f6640f0d-wqhf5"]
Sep 08 14:10:50.955760 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:50.954743    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:50.957678 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:50.957566    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:50.971347 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:50.971320    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"]
Sep 08 14:10:50.971537 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:50.971514    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy" containerID="cri-o://bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398" gracePeriod=30
Sep 08 14:10:51.075702 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.075664    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmgvc\" (UniqueName: \"kubernetes.io/projected/82ca4424-326e-4fd7-8a64-7d8712aa8b07-kube-api-access-kmgvc\") pod \"82ca4424-326e-4fd7-8a64-7d8712aa8b07\" (UID: \"82ca4424-326e-4fd7-8a64-7d8712aa8b07\") "
Sep 08 14:10:51.075903 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.075743    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82ca4424-326e-4fd7-8a64-7d8712aa8b07-plugins-config-volume\") pod \"82ca4424-326e-4fd7-8a64-7d8712aa8b07\" (UID: \"82ca4424-326e-4fd7-8a64-7d8712aa8b07\") "
Sep 08 14:10:51.075903 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.075793    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x9lm\" (UniqueName: \"kubernetes.io/projected/98075290-621f-4477-8123-4017c06f943b-kube-api-access-2x9lm\") pod \"98075290-621f-4477-8123-4017c06f943b\" (UID: \"98075290-621f-4477-8123-4017c06f943b\") "
Sep 08 14:10:51.075903 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.075839    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98075290-621f-4477-8123-4017c06f943b-plugins-config-volume\") pod \"98075290-621f-4477-8123-4017c06f943b\" (UID: \"98075290-621f-4477-8123-4017c06f943b\") "
Sep 08 14:10:51.076128 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.076100    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ca4424-326e-4fd7-8a64-7d8712aa8b07-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82ca4424-326e-4fd7-8a64-7d8712aa8b07" (UID: "82ca4424-326e-4fd7-8a64-7d8712aa8b07"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:10:51.076328 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.076256    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98075290-621f-4477-8123-4017c06f943b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "98075290-621f-4477-8123-4017c06f943b" (UID: "98075290-621f-4477-8123-4017c06f943b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:10:51.076477 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.076458    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82ca4424-326e-4fd7-8a64-7d8712aa8b07-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:51.076537 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.076501    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98075290-621f-4477-8123-4017c06f943b-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:51.078836 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.078807    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ca4424-326e-4fd7-8a64-7d8712aa8b07-kube-api-access-kmgvc" (OuterVolumeSpecName: "kube-api-access-kmgvc") pod "82ca4424-326e-4fd7-8a64-7d8712aa8b07" (UID: "82ca4424-326e-4fd7-8a64-7d8712aa8b07"). InnerVolumeSpecName "kube-api-access-kmgvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:10:51.079183 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.079164    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98075290-621f-4477-8123-4017c06f943b-kube-api-access-2x9lm" (OuterVolumeSpecName: "kube-api-access-2x9lm") pod "98075290-621f-4477-8123-4017c06f943b" (UID: "98075290-621f-4477-8123-4017c06f943b"). InnerVolumeSpecName "kube-api-access-2x9lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:10:51.177956 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.177850    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmgvc\" (UniqueName: \"kubernetes.io/projected/82ca4424-326e-4fd7-8a64-7d8712aa8b07-kube-api-access-kmgvc\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:51.177956 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.177902    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x9lm\" (UniqueName: \"kubernetes.io/projected/98075290-621f-4477-8123-4017c06f943b-kube-api-access-2x9lm\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:51.446812 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.446721    2564 generic.go:356] "Generic (PLEG): container finished" podID="82ca4424-326e-4fd7-8a64-7d8712aa8b07" containerID="568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad" exitCode=0
Sep 08 14:10:51.446812 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.446778    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"
Sep 08 14:10:51.446812 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.446786    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl" event={"ID":"82ca4424-326e-4fd7-8a64-7d8712aa8b07","Type":"ContainerDied","Data":"568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad"}
Sep 08 14:10:51.447116 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.446848    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl" event={"ID":"82ca4424-326e-4fd7-8a64-7d8712aa8b07","Type":"ContainerDied","Data":"6eafbdc1acb9ee304bcbdbc7a9181db1ae0e4d853093c240def599a641a33f90"}
Sep 08 14:10:51.447116 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.446890    2564 scope.go:117] "RemoveContainer" containerID="568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad"
Sep 08 14:10:51.448372 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.448350    2564 generic.go:356] "Generic (PLEG): container finished" podID="98075290-621f-4477-8123-4017c06f943b" containerID="eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf" exitCode=0
Sep 08 14:10:51.448464 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.448388    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"
Sep 08 14:10:51.448464 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.448440    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq" event={"ID":"98075290-621f-4477-8123-4017c06f943b","Type":"ContainerDied","Data":"eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf"}
Sep 08 14:10:51.448576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.448494    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq" event={"ID":"98075290-621f-4477-8123-4017c06f943b","Type":"ContainerDied","Data":"63cbf58041fc3ef0bbbfd6014a8dc22b20020287004e39b7a5e25c42c301ecb7"}
Sep 08 14:10:51.457144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.457126    2564 scope.go:117] "RemoveContainer" containerID="568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad"
Sep 08 14:10:51.457474 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:10:51.457450    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad\": container with ID starting with 568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad not found: ID does not exist" containerID="568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad"
Sep 08 14:10:51.457560 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.457480    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad"} err="failed to get container status \"568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad\": rpc error: code = NotFound desc = could not find container \"568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad\": container with ID starting with 568f72dcca6d58ce0e6cab78c41a31a19f47939cd9394dff377a10f7575c41ad not found: ID does not exist"
Sep 08 14:10:51.457560 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.457497    2564 scope.go:117] "RemoveContainer" containerID="eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf"
Sep 08 14:10:51.465972 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.465953    2564 scope.go:117] "RemoveContainer" containerID="eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf"
Sep 08 14:10:51.466217 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:10:51.466194    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf\": container with ID starting with eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf not found: ID does not exist" containerID="eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf"
Sep 08 14:10:51.466311 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.466225    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf"} err="failed to get container status \"eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf\": rpc error: code = NotFound desc = could not find container \"eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf\": container with ID starting with eac618e849985d2329a757fd2b92fd81080e4c3e9a67144914703f343f7b47bf not found: ID does not exist"
Sep 08 14:10:51.472614 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.472592    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"]
Sep 08 14:10:51.476110 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.476091    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f6640f0d-554c9cb79d-tnnwq"]
Sep 08 14:10:51.487239 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.487212    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"]
Sep 08 14:10:51.491685 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.491657    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f6640f0d-5dc7d5cd4b-w9kvl"]
Sep 08 14:10:51.960116 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.960084    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7259b36e-f673-431b-ba1b-05131cd625f3" path="/var/lib/kubelet/pods/7259b36e-f673-431b-ba1b-05131cd625f3/volumes"
Sep 08 14:10:51.960473 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.960460    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82ca4424-326e-4fd7-8a64-7d8712aa8b07" path="/var/lib/kubelet/pods/82ca4424-326e-4fd7-8a64-7d8712aa8b07/volumes"
Sep 08 14:10:51.960786 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:51.960773    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98075290-621f-4477-8123-4017c06f943b" path="/var/lib/kubelet/pods/98075290-621f-4477-8123-4017c06f943b/volumes"
Sep 08 14:10:56.234826 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.234756    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:56.329031 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.328994    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ca42b31-9962-4880-98d9-67f712cd5c5f-istiod-ca-cert\") pod \"1ca42b31-9962-4880-98d9-67f712cd5c5f\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") "
Sep 08 14:10:56.329225 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329054    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crqm6\" (UniqueName: \"kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-kube-api-access-crqm6\") pod \"1ca42b31-9962-4880-98d9-67f712cd5c5f\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") "
Sep 08 14:10:56.329225 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329165    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-certs\") pod \"1ca42b31-9962-4880-98d9-67f712cd5c5f\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") "
Sep 08 14:10:56.329225 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329204    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-token\") pod \"1ca42b31-9962-4880-98d9-67f712cd5c5f\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") "
Sep 08 14:10:56.329411 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329280    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-podinfo\") pod \"1ca42b31-9962-4880-98d9-67f712cd5c5f\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") "
Sep 08 14:10:56.329411 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329319    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-socket\") pod \"1ca42b31-9962-4880-98d9-67f712cd5c5f\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") "
Sep 08 14:10:56.329411 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329344    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-data\") pod \"1ca42b31-9962-4880-98d9-67f712cd5c5f\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") "
Sep 08 14:10:56.329411 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329381    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-credential-socket\") pod \"1ca42b31-9962-4880-98d9-67f712cd5c5f\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") "
Sep 08 14:10:56.329411 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329379    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ca42b31-9962-4880-98d9-67f712cd5c5f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ca42b31-9962-4880-98d9-67f712cd5c5f" (UID: "1ca42b31-9962-4880-98d9-67f712cd5c5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:10:56.329666 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329413    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ca42b31-9962-4880-98d9-67f712cd5c5f" (UID: "1ca42b31-9962-4880-98d9-67f712cd5c5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:10:56.329666 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329483    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-envoy\") pod \"1ca42b31-9962-4880-98d9-67f712cd5c5f\" (UID: \"1ca42b31-9962-4880-98d9-67f712cd5c5f\") "
Sep 08 14:10:56.329666 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329542    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ca42b31-9962-4880-98d9-67f712cd5c5f" (UID: "1ca42b31-9962-4880-98d9-67f712cd5c5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:10:56.329666 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329589    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ca42b31-9962-4880-98d9-67f712cd5c5f" (UID: "1ca42b31-9962-4880-98d9-67f712cd5c5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:10:56.329895 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329691    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ca42b31-9962-4880-98d9-67f712cd5c5f" (UID: "1ca42b31-9962-4880-98d9-67f712cd5c5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:10:56.329961 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329890    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ca42b31-9962-4880-98d9-67f712cd5c5f-istiod-ca-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:56.329961 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329911    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:56.329961 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329924    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-workload-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:56.329961 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329936    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-data\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:56.329961 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.329948    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-credential-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:56.331901 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.331838    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ca42b31-9962-4880-98d9-67f712cd5c5f" (UID: "1ca42b31-9962-4880-98d9-67f712cd5c5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:10:56.331901 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.331848    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-kube-api-access-crqm6" (OuterVolumeSpecName: "kube-api-access-crqm6") pod "1ca42b31-9962-4880-98d9-67f712cd5c5f" (UID: "1ca42b31-9962-4880-98d9-67f712cd5c5f"). InnerVolumeSpecName "kube-api-access-crqm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:10:56.332099 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.331969    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ca42b31-9962-4880-98d9-67f712cd5c5f" (UID: "1ca42b31-9962-4880-98d9-67f712cd5c5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:10:56.332155 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.332096    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ca42b31-9962-4880-98d9-67f712cd5c5f" (UID: "1ca42b31-9962-4880-98d9-67f712cd5c5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:10:56.381904 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.381856    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"]
Sep 08 14:10:56.382255 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382244    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy"
Sep 08 14:10:56.382301 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382257    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy"
Sep 08 14:10:56.382301 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382269    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy"
Sep 08 14:10:56.382301 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382274    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy"
Sep 08 14:10:56.382301 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382283    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7259b36e-f673-431b-ba1b-05131cd625f3" containerName="revoke-keys"
Sep 08 14:10:56.382301 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382289    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7259b36e-f673-431b-ba1b-05131cd625f3" containerName="revoke-keys"
Sep 08 14:10:56.382301 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382299    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98075290-621f-4477-8123-4017c06f943b" containerName="payload-pre-processing"
Sep 08 14:10:56.382301 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382304    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="98075290-621f-4477-8123-4017c06f943b" containerName="payload-pre-processing"
Sep 08 14:10:56.382519 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382317    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ca4424-326e-4fd7-8a64-7d8712aa8b07" containerName="payload-processing"
Sep 08 14:10:56.382519 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382323    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ca4424-326e-4fd7-8a64-7d8712aa8b07" containerName="payload-processing"
Sep 08 14:10:56.382519 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382381    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="98075290-621f-4477-8123-4017c06f943b" containerName="payload-pre-processing"
Sep 08 14:10:56.382519 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382387    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ca4424-326e-4fd7-8a64-7d8712aa8b07" containerName="payload-processing"
Sep 08 14:10:56.382519 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382394    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7259b36e-f673-431b-ba1b-05131cd625f3" containerName="revoke-keys"
Sep 08 14:10:56.382519 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382400    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy"
Sep 08 14:10:56.382519 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.382408    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerName="istio-proxy"
Sep 08 14:10:56.384425 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.384410    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:10:56.387234 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.387208    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-af7a8d88\""
Sep 08 14:10:56.387379 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.387291    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-af7a8d88-dockercfg-pswc7\""
Sep 08 14:10:56.399793 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.399767    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"]
Sep 08 14:10:56.430880 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.430828    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-podinfo\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:56.431049 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.430900    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-envoy\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:56.431049 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.430915    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crqm6\" (UniqueName: \"kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-kube-api-access-crqm6\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:56.431049 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.430928    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ca42b31-9962-4880-98d9-67f712cd5c5f-istio-token\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:10:56.472575 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.472542    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"]
Sep 08 14:10:56.475197 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.475172    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:10:56.482205 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.482078    2564 generic.go:356] "Generic (PLEG): container finished" podID="1ca42b31-9962-4880-98d9-67f712cd5c5f" containerID="bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398" exitCode=0
Sep 08 14:10:56.482205 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.482151    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"
Sep 08 14:10:56.482205 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.482161    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" event={"ID":"1ca42b31-9962-4880-98d9-67f712cd5c5f","Type":"ContainerDied","Data":"bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398"}
Sep 08 14:10:56.482563 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.482210    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf" event={"ID":"1ca42b31-9962-4880-98d9-67f712cd5c5f","Type":"ContainerDied","Data":"e5bd5b3e048729361f0e20366e7ea58077e34212f79563162d6c376cc730a356"}
Sep 08 14:10:56.482563 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.482244    2564 scope.go:117] "RemoveContainer" containerID="bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398"
Sep 08 14:10:56.493457 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.493423    2564 scope.go:117] "RemoveContainer" containerID="24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd"
Sep 08 14:10:56.495249 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.495225    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"]
Sep 08 14:10:56.504223 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.504204    2564 scope.go:117] "RemoveContainer" containerID="bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398"
Sep 08 14:10:56.504509 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:10:56.504490    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398\": container with ID starting with bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398 not found: ID does not exist" containerID="bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398"
Sep 08 14:10:56.504585 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.504515    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398"} err="failed to get container status \"bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398\": rpc error: code = NotFound desc = could not find container \"bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398\": container with ID starting with bc8d901e0e97e8ab577aee41219b6ddb419cccb66bade4b55ad27583135f4398 not found: ID does not exist"
Sep 08 14:10:56.504585 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.504532    2564 scope.go:117] "RemoveContainer" containerID="24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd"
Sep 08 14:10:56.504780 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:10:56.504761    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd\": container with ID starting with 24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd not found: ID does not exist" containerID="24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd"
Sep 08 14:10:56.504886 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.504780    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd"} err="failed to get container status \"24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd\": rpc error: code = NotFound desc = could not find container \"24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd\": container with ID starting with 24b2cee6b35435a53c604e3e3ae35e73f53b2e6bb15ef0b9460236c09bd23cfd not found: ID does not exist"
Sep 08 14:10:56.532457 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.532409    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8p97\" (UniqueName: \"kubernetes.io/projected/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-kube-api-access-k8p97\") pod \"payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9\" (UID: \"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:10:56.532744 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.532719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9\" (UID: \"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:10:56.533103 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.532586    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"]
Sep 08 14:10:56.535219 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.535198    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-f6640f0d-gw-openshift-default-65b6cdf5b8-zp8qf"]
Sep 08 14:10:56.634148 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.634100    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-plugins-config-volume\") pod \"payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf\" (UID: \"5cb2862c-3c13-4dd0-971e-78fd2ce77ece\") " pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:10:56.634351 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.634173    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8p97\" (UniqueName: \"kubernetes.io/projected/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-kube-api-access-k8p97\") pod \"payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9\" (UID: \"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:10:56.634351 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.634222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9\" (UID: \"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:10:56.634467 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.634350    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q5n7\" (UniqueName: \"kubernetes.io/projected/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-kube-api-access-7q5n7\") pod \"payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf\" (UID: \"5cb2862c-3c13-4dd0-971e-78fd2ce77ece\") " pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:10:56.635015 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.634993    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9\" (UID: \"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:10:56.643965 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.643927    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8p97\" (UniqueName: \"kubernetes.io/projected/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-kube-api-access-k8p97\") pod \"payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9\" (UID: \"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:10:56.695346 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.695307    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:10:56.735663 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.735629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q5n7\" (UniqueName: \"kubernetes.io/projected/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-kube-api-access-7q5n7\") pod \"payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf\" (UID: \"5cb2862c-3c13-4dd0-971e-78fd2ce77ece\") " pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:10:56.735831 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.735687    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-plugins-config-volume\") pod \"payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf\" (UID: \"5cb2862c-3c13-4dd0-971e-78fd2ce77ece\") " pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:10:56.736419 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.736399    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-plugins-config-volume\") pod \"payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf\" (UID: \"5cb2862c-3c13-4dd0-971e-78fd2ce77ece\") " pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:10:56.745625 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.745600    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q5n7\" (UniqueName: \"kubernetes.io/projected/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-kube-api-access-7q5n7\") pod \"payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf\" (UID: \"5cb2862c-3c13-4dd0-971e-78fd2ce77ece\") " pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:10:56.787058 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.787006    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:10:56.968727 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:56.968700    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"]
Sep 08 14:10:56.970933 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:10:56.970901    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1ebe4b_fc71_49bb_abc1_d8e266ed23a1.slice/crio-df596efc641ad3d69a9225b111bad3a171f9d1bfcfe5286c63ae5d32542c79ed WatchSource:0}: Error finding container df596efc641ad3d69a9225b111bad3a171f9d1bfcfe5286c63ae5d32542c79ed: Status 404 returned error can't find the container with id df596efc641ad3d69a9225b111bad3a171f9d1bfcfe5286c63ae5d32542c79ed
Sep 08 14:10:57.074345 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:57.074212    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"]
Sep 08 14:10:57.077081 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:10:57.077050    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb2862c_3c13_4dd0_971e_78fd2ce77ece.slice/crio-9fc1cf022eceb28a33c606735b66f1ffa5028e2d02a81ece683824b1f9334eb4 WatchSource:0}: Error finding container 9fc1cf022eceb28a33c606735b66f1ffa5028e2d02a81ece683824b1f9334eb4: Status 404 returned error can't find the container with id 9fc1cf022eceb28a33c606735b66f1ffa5028e2d02a81ece683824b1f9334eb4
Sep 08 14:10:57.489593 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:57.489555    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf" event={"ID":"5cb2862c-3c13-4dd0-971e-78fd2ce77ece","Type":"ContainerStarted","Data":"109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7"}
Sep 08 14:10:57.490059 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:57.489619    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf" event={"ID":"5cb2862c-3c13-4dd0-971e-78fd2ce77ece","Type":"ContainerStarted","Data":"9fc1cf022eceb28a33c606735b66f1ffa5028e2d02a81ece683824b1f9334eb4"}
Sep 08 14:10:57.490059 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:57.489664    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:10:57.491256 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:57.491236    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9" event={"ID":"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1","Type":"ContainerStarted","Data":"7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f"}
Sep 08 14:10:57.491341 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:57.491286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9" event={"ID":"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1","Type":"ContainerStarted","Data":"df596efc641ad3d69a9225b111bad3a171f9d1bfcfe5286c63ae5d32542c79ed"}
Sep 08 14:10:57.491341 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:57.491308    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:10:57.507519 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:57.507474    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf" podStartSLOduration=1.507460478 podStartE2EDuration="1.507460478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:10:57.505764134 +0000 UTC m=+748.000734829" watchObservedRunningTime="2026-09-08 14:10:57.507460478 +0000 UTC m=+748.002431152"
Sep 08 14:10:57.523728 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:57.523688    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9" podStartSLOduration=1.523672215 podStartE2EDuration="1.523672215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:10:57.522035345 +0000 UTC m=+748.017006029" watchObservedRunningTime="2026-09-08 14:10:57.523672215 +0000 UTC m=+748.018642900"
Sep 08 14:10:57.960054 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:10:57.959973    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ca42b31-9962-4880-98d9-67f712cd5c5f" path="/var/lib/kubelet/pods/1ca42b31-9962-4880-98d9-67f712cd5c5f/volumes"
Sep 08 14:11:08.499222 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:08.499184    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:11:08.499661 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:08.499281    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:11:20.165156 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:20.165115    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"]
Sep 08 14:11:20.168841 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:20.168822    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"
Sep 08 14:11:20.175835 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:20.175427    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:11:20.175835 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:20.175435    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:11:20.183810 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:20.183785    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"]
Sep 08 14:11:20.270930 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:20.270892    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fec02f14-cc8a-43cf-ae8a-5323982c809d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr\" (UID: \"fec02f14-cc8a-43cf-ae8a-5323982c809d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"
Sep 08 14:11:20.372078 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:20.372044    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fec02f14-cc8a-43cf-ae8a-5323982c809d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr\" (UID: \"fec02f14-cc8a-43cf-ae8a-5323982c809d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"
Sep 08 14:11:20.372665 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:20.372645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fec02f14-cc8a-43cf-ae8a-5323982c809d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr\" (UID: \"fec02f14-cc8a-43cf-ae8a-5323982c809d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"
Sep 08 14:11:20.479271 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:20.479242    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"
Sep 08 14:11:20.765560 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:20.765534    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"]
Sep 08 14:11:20.767915 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:11:20.767884    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfec02f14_cc8a_43cf_ae8a_5323982c809d.slice/crio-4be6e68107f006b341eca632741640274e6e458950024db35e30b5b5c47d4f3e WatchSource:0}: Error finding container 4be6e68107f006b341eca632741640274e6e458950024db35e30b5b5c47d4f3e: Status 404 returned error can't find the container with id 4be6e68107f006b341eca632741640274e6e458950024db35e30b5b5c47d4f3e
Sep 08 14:11:21.624541 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:21.624503    2564 generic.go:356] "Generic (PLEG): container finished" podID="fec02f14-cc8a-43cf-ae8a-5323982c809d" containerID="b25ce01366e833d18d49e640f6680833f1d4c2a609248729cfcaea7f6b208855" exitCode=0
Sep 08 14:11:21.624940 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:21.624600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr" event={"ID":"fec02f14-cc8a-43cf-ae8a-5323982c809d","Type":"ContainerDied","Data":"b25ce01366e833d18d49e640f6680833f1d4c2a609248729cfcaea7f6b208855"}
Sep 08 14:11:21.624940 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:21.624669    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr" event={"ID":"fec02f14-cc8a-43cf-ae8a-5323982c809d","Type":"ContainerStarted","Data":"4be6e68107f006b341eca632741640274e6e458950024db35e30b5b5c47d4f3e"}
Sep 08 14:11:22.771141 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:22.771106    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"
Sep 08 14:11:22.898183 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:22.898102    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fec02f14-cc8a-43cf-ae8a-5323982c809d-maas-api-service-ca\") pod \"fec02f14-cc8a-43cf-ae8a-5323982c809d\" (UID: \"fec02f14-cc8a-43cf-ae8a-5323982c809d\") "
Sep 08 14:11:22.898530 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:22.898502    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fec02f14-cc8a-43cf-ae8a-5323982c809d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fec02f14-cc8a-43cf-ae8a-5323982c809d" (UID: "fec02f14-cc8a-43cf-ae8a-5323982c809d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:11:22.999115 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:22.999074    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fec02f14-cc8a-43cf-ae8a-5323982c809d-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:11:23.643946 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:23.643914    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"
Sep 08 14:11:23.643946 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:23.643938    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr" event={"ID":"fec02f14-cc8a-43cf-ae8a-5323982c809d","Type":"ContainerDied","Data":"4be6e68107f006b341eca632741640274e6e458950024db35e30b5b5c47d4f3e"}
Sep 08 14:11:23.644162 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:23.643987    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4be6e68107f006b341eca632741640274e6e458950024db35e30b5b5c47d4f3e"
Sep 08 14:11:26.174624 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.174584    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"]
Sep 08 14:11:26.175018 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.174814    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf" podUID="5cb2862c-3c13-4dd0-971e-78fd2ce77ece" containerName="payload-processing" containerID="cri-o://109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7" gracePeriod=30
Sep 08 14:11:26.200936 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.200910    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"]
Sep 08 14:11:26.201108 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.201089    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9" podUID="ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1" containerName="payload-pre-processing" containerID="cri-o://7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f" gracePeriod=30
Sep 08 14:11:26.437027 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.436944    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"]
Sep 08 14:11:26.439694 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.439670    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-af7a8d88-2dqcr"]
Sep 08 14:11:26.482979 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.482958    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:11:26.485935 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.485918    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:11:26.635981 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.635949    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-plugins-config-volume\") pod \"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1\" (UID: \"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1\") "
Sep 08 14:11:26.636169 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.636009    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q5n7\" (UniqueName: \"kubernetes.io/projected/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-kube-api-access-7q5n7\") pod \"5cb2862c-3c13-4dd0-971e-78fd2ce77ece\" (UID: \"5cb2862c-3c13-4dd0-971e-78fd2ce77ece\") "
Sep 08 14:11:26.636169 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.636050    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-plugins-config-volume\") pod \"5cb2862c-3c13-4dd0-971e-78fd2ce77ece\" (UID: \"5cb2862c-3c13-4dd0-971e-78fd2ce77ece\") "
Sep 08 14:11:26.636169 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.636084    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8p97\" (UniqueName: \"kubernetes.io/projected/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-kube-api-access-k8p97\") pod \"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1\" (UID: \"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1\") "
Sep 08 14:11:26.636373 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.636344    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1" (UID: "ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:11:26.636487 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.636463    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5cb2862c-3c13-4dd0-971e-78fd2ce77ece" (UID: "5cb2862c-3c13-4dd0-971e-78fd2ce77ece"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:11:26.638426 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.638402    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-kube-api-access-k8p97" (OuterVolumeSpecName: "kube-api-access-k8p97") pod "ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1" (UID: "ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1"). InnerVolumeSpecName "kube-api-access-k8p97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:11:26.638510 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.638465    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-kube-api-access-7q5n7" (OuterVolumeSpecName: "kube-api-access-7q5n7") pod "5cb2862c-3c13-4dd0-971e-78fd2ce77ece" (UID: "5cb2862c-3c13-4dd0-971e-78fd2ce77ece"). InnerVolumeSpecName "kube-api-access-7q5n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:11:26.661269 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.661240    2564 generic.go:356] "Generic (PLEG): container finished" podID="5cb2862c-3c13-4dd0-971e-78fd2ce77ece" containerID="109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7" exitCode=0
Sep 08 14:11:26.661377 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.661282    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf" event={"ID":"5cb2862c-3c13-4dd0-971e-78fd2ce77ece","Type":"ContainerDied","Data":"109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7"}
Sep 08 14:11:26.661377 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.661295    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"
Sep 08 14:11:26.661377 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.661332    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf" event={"ID":"5cb2862c-3c13-4dd0-971e-78fd2ce77ece","Type":"ContainerDied","Data":"9fc1cf022eceb28a33c606735b66f1ffa5028e2d02a81ece683824b1f9334eb4"}
Sep 08 14:11:26.661377 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.661354    2564 scope.go:117] "RemoveContainer" containerID="109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7"
Sep 08 14:11:26.662796 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.662775    2564 generic.go:356] "Generic (PLEG): container finished" podID="ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1" containerID="7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f" exitCode=0
Sep 08 14:11:26.662921 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.662820    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"
Sep 08 14:11:26.662921 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.662841    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9" event={"ID":"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1","Type":"ContainerDied","Data":"7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f"}
Sep 08 14:11:26.663019 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.662936    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9" event={"ID":"ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1","Type":"ContainerDied","Data":"df596efc641ad3d69a9225b111bad3a171f9d1bfcfe5286c63ae5d32542c79ed"}
Sep 08 14:11:26.671917 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.671856    2564 scope.go:117] "RemoveContainer" containerID="109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7"
Sep 08 14:11:26.672209 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:11:26.672187    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7\": container with ID starting with 109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7 not found: ID does not exist" containerID="109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7"
Sep 08 14:11:26.672294 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.672217    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7"} err="failed to get container status \"109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7\": rpc error: code = NotFound desc = could not find container \"109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7\": container with ID starting with 109505c04c4a0410f9292c6bc47960bc22e7e639712082ba8ba3f67fcb9411f7 not found: ID does not exist"
Sep 08 14:11:26.672294 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.672234    2564 scope.go:117] "RemoveContainer" containerID="7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f"
Sep 08 14:11:26.680420 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.680404    2564 scope.go:117] "RemoveContainer" containerID="7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f"
Sep 08 14:11:26.680705 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:11:26.680688    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f\": container with ID starting with 7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f not found: ID does not exist" containerID="7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f"
Sep 08 14:11:26.680778 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.680709    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f"} err="failed to get container status \"7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f\": rpc error: code = NotFound desc = could not find container \"7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f\": container with ID starting with 7c07a6bb737ae9bcca0eeefc4628aa3e90def754fcbd59d18c0820577f83c86f not found: ID does not exist"
Sep 08 14:11:26.702543 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.702477    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"]
Sep 08 14:11:26.708272 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.708252    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af7a8d88-6c6f7f7446-2zrd9"]
Sep 08 14:11:26.721989 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.721966    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"]
Sep 08 14:11:26.726692 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.726671    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af7a8d88-7df7474585-nrfrf"]
Sep 08 14:11:26.737173 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.737153    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:11:26.737257 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.737178    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q5n7\" (UniqueName: \"kubernetes.io/projected/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-kube-api-access-7q5n7\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:11:26.737257 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.737189    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cb2862c-3c13-4dd0-971e-78fd2ce77ece-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:11:26.737257 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:26.737197    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8p97\" (UniqueName: \"kubernetes.io/projected/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1-kube-api-access-k8p97\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:11:27.959346 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:27.959308    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cb2862c-3c13-4dd0-971e-78fd2ce77ece" path="/var/lib/kubelet/pods/5cb2862c-3c13-4dd0-971e-78fd2ce77ece/volumes"
Sep 08 14:11:27.959985 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:27.959814    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1" path="/var/lib/kubelet/pods/ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1/volumes"
Sep 08 14:11:27.960239 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:27.960218    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fec02f14-cc8a-43cf-ae8a-5323982c809d" path="/var/lib/kubelet/pods/fec02f14-cc8a-43cf-ae8a-5323982c809d/volumes"
Sep 08 14:11:28.552073 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.552040    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"]
Sep 08 14:11:28.552447 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.552434    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fec02f14-cc8a-43cf-ae8a-5323982c809d" containerName="revoke-keys"
Sep 08 14:11:28.552494 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.552448    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fec02f14-cc8a-43cf-ae8a-5323982c809d" containerName="revoke-keys"
Sep 08 14:11:28.552494 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.552457    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb2862c-3c13-4dd0-971e-78fd2ce77ece" containerName="payload-processing"
Sep 08 14:11:28.552494 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.552463    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb2862c-3c13-4dd0-971e-78fd2ce77ece" containerName="payload-processing"
Sep 08 14:11:28.552494 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.552486    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1" containerName="payload-pre-processing"
Sep 08 14:11:28.552494 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.552492    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1" containerName="payload-pre-processing"
Sep 08 14:11:28.552650 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.552552    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac1ebe4b-fc71-49bb-abc1-d8e266ed23a1" containerName="payload-pre-processing"
Sep 08 14:11:28.552650 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.552560    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fec02f14-cc8a-43cf-ae8a-5323982c809d" containerName="revoke-keys"
Sep 08 14:11:28.552650 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.552565    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb2862c-3c13-4dd0-971e-78fd2ce77ece" containerName="payload-processing"
Sep 08 14:11:28.557778 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.557758    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.565822 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.565803    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-93eb17b6-openshift-default-dockercfg-nfj9p\""
Sep 08 14:11:28.579164 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.579138    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"]
Sep 08 14:11:28.655418 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.655387    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-certs\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.655418 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.655430    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-credential-socket\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.655689 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.655509    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gndpx\" (UniqueName: \"kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-kube-api-access-gndpx\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.655689 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.655566    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-data\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.655689 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.655602    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8412890e-56a7-4b8f-b127-070e34897640-istio-podinfo\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.655689 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.655623    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-envoy\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.656059 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.655729    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-socket\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.656059 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.655789    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8412890e-56a7-4b8f-b127-070e34897640-istiod-ca-cert\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.656059 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.655820    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-istio-token\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.757424 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757387    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-credential-socket\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.757424 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757432    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gndpx\" (UniqueName: \"kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-kube-api-access-gndpx\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.757668 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757460    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-data\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.757668 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757483    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8412890e-56a7-4b8f-b127-070e34897640-istio-podinfo\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.757668 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757501    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-envoy\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.757668 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757521    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-socket\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.757668 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757557    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8412890e-56a7-4b8f-b127-070e34897640-istiod-ca-cert\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.757668 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757584    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-istio-token\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.757668 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-certs\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.758084 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757843    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-credential-socket\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.758084 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757919    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-data\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.758084 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.757985    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-socket\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.758236 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.758217    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-certs\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.758369 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.758352    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8412890e-56a7-4b8f-b127-070e34897640-istiod-ca-cert\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.760026 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.759992    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-envoy\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.760172 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.760155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8412890e-56a7-4b8f-b127-070e34897640-istio-podinfo\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.768761 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.768737    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-istio-token\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.769193 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.769176    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gndpx\" (UniqueName: \"kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-kube-api-access-gndpx\") pod \"e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") " pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:28.868696 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:28.868607    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:29.169209 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:29.169183    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"]
Sep 08 14:11:29.170927 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:11:29.170900    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8412890e_56a7_4b8f_b127_070e34897640.slice/crio-86121a8a4c09a8463cc23208379fe2e935e6cdd5ad1adcbba54e040acbfeb6c6 WatchSource:0}: Error finding container 86121a8a4c09a8463cc23208379fe2e935e6cdd5ad1adcbba54e040acbfeb6c6: Status 404 returned error can't find the container with id 86121a8a4c09a8463cc23208379fe2e935e6cdd5ad1adcbba54e040acbfeb6c6
Sep 08 14:11:29.173157 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:29.173122    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:11:29.173226 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:29.173198    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:11:29.173264 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:29.173227    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:11:29.681896 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:29.681835    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx" event={"ID":"8412890e-56a7-4b8f-b127-070e34897640","Type":"ContainerStarted","Data":"6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e"}
Sep 08 14:11:29.682108 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:29.681912    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx" event={"ID":"8412890e-56a7-4b8f-b127-070e34897640","Type":"ContainerStarted","Data":"86121a8a4c09a8463cc23208379fe2e935e6cdd5ad1adcbba54e040acbfeb6c6"}
Sep 08 14:11:29.708839 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:29.708776    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx" podStartSLOduration=1.708755559 podStartE2EDuration="1.708755559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:11:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:11:29.708096843 +0000 UTC m=+780.203067519" watchObservedRunningTime="2026-09-08 14:11:29.708755559 +0000 UTC m=+780.203726244"
Sep 08 14:11:29.868953 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:29.868911    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:30.875207 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:30.875176    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:31.691623 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:31.691591    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:31.693176 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:31.693154    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:11:33.218329 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.218294    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"]
Sep 08 14:11:33.223402 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.223369    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:11:33.227149 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.227126    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-93eb17b6\""
Sep 08 14:11:33.234912 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.234845    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"]
Sep 08 14:11:33.250227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.250203    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"]
Sep 08 14:11:33.254421 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.254399    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:11:33.256944 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.256901    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-93eb17b6-dockercfg-nbr6n\""
Sep 08 14:11:33.264819 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.264798    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"]
Sep 08 14:11:33.408348 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.408308    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpl27\" (UniqueName: \"kubernetes.io/projected/55107102-3f5b-4698-9677-e4a191f2d5a0-kube-api-access-xpl27\") pod \"payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt\" (UID: \"55107102-3f5b-4698-9677-e4a191f2d5a0\") " pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:11:33.408514 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.408438    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg4sw\" (UniqueName: \"kubernetes.io/projected/7bc71734-6429-4435-bbc4-284839fc2621-kube-api-access-mg4sw\") pod \"payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x\" (UID: \"7bc71734-6429-4435-bbc4-284839fc2621\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:11:33.408514 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.408492    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55107102-3f5b-4698-9677-e4a191f2d5a0-plugins-config-volume\") pod \"payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt\" (UID: \"55107102-3f5b-4698-9677-e4a191f2d5a0\") " pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:11:33.408631 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.408547    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bc71734-6429-4435-bbc4-284839fc2621-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x\" (UID: \"7bc71734-6429-4435-bbc4-284839fc2621\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:11:33.509713 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.509625    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg4sw\" (UniqueName: \"kubernetes.io/projected/7bc71734-6429-4435-bbc4-284839fc2621-kube-api-access-mg4sw\") pod \"payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x\" (UID: \"7bc71734-6429-4435-bbc4-284839fc2621\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:11:33.509713 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.509668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55107102-3f5b-4698-9677-e4a191f2d5a0-plugins-config-volume\") pod \"payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt\" (UID: \"55107102-3f5b-4698-9677-e4a191f2d5a0\") " pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:11:33.509713 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.509691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bc71734-6429-4435-bbc4-284839fc2621-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x\" (UID: \"7bc71734-6429-4435-bbc4-284839fc2621\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:11:33.510028 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.509757    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpl27\" (UniqueName: \"kubernetes.io/projected/55107102-3f5b-4698-9677-e4a191f2d5a0-kube-api-access-xpl27\") pod \"payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt\" (UID: \"55107102-3f5b-4698-9677-e4a191f2d5a0\") " pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:11:33.510599 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.510575    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55107102-3f5b-4698-9677-e4a191f2d5a0-plugins-config-volume\") pod \"payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt\" (UID: \"55107102-3f5b-4698-9677-e4a191f2d5a0\") " pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:11:33.510643 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.510575    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bc71734-6429-4435-bbc4-284839fc2621-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x\" (UID: \"7bc71734-6429-4435-bbc4-284839fc2621\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:11:33.518413 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.518389    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpl27\" (UniqueName: \"kubernetes.io/projected/55107102-3f5b-4698-9677-e4a191f2d5a0-kube-api-access-xpl27\") pod \"payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt\" (UID: \"55107102-3f5b-4698-9677-e4a191f2d5a0\") " pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:11:33.518585 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.518453    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg4sw\" (UniqueName: \"kubernetes.io/projected/7bc71734-6429-4435-bbc4-284839fc2621-kube-api-access-mg4sw\") pod \"payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x\" (UID: \"7bc71734-6429-4435-bbc4-284839fc2621\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:11:33.535241 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.535216    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:11:33.566407 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.566375    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:11:33.814243 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.814220    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"]
Sep 08 14:11:33.816141 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:11:33.816111    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc71734_6429_4435_bbc4_284839fc2621.slice/crio-b9a54d3d72deb5a0f486795e4eaca6f8fb55812411103695d149c958bfd2d640 WatchSource:0}: Error finding container b9a54d3d72deb5a0f486795e4eaca6f8fb55812411103695d149c958bfd2d640: Status 404 returned error can't find the container with id b9a54d3d72deb5a0f486795e4eaca6f8fb55812411103695d149c958bfd2d640
Sep 08 14:11:33.850995 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:33.850971    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"]
Sep 08 14:11:33.852738 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:11:33.852710    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55107102_3f5b_4698_9677_e4a191f2d5a0.slice/crio-bf3a96ddc98511647bef525e4bcfdb062e55b27f5c6b2040d2d68694edeb58f3 WatchSource:0}: Error finding container bf3a96ddc98511647bef525e4bcfdb062e55b27f5c6b2040d2d68694edeb58f3: Status 404 returned error can't find the container with id bf3a96ddc98511647bef525e4bcfdb062e55b27f5c6b2040d2d68694edeb58f3
Sep 08 14:11:34.708623 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:34.708584    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt" event={"ID":"55107102-3f5b-4698-9677-e4a191f2d5a0","Type":"ContainerStarted","Data":"946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e"}
Sep 08 14:11:34.709126 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:34.708644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt" event={"ID":"55107102-3f5b-4698-9677-e4a191f2d5a0","Type":"ContainerStarted","Data":"bf3a96ddc98511647bef525e4bcfdb062e55b27f5c6b2040d2d68694edeb58f3"}
Sep 08 14:11:34.709126 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:34.708766    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:11:34.710240 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:34.710218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x" event={"ID":"7bc71734-6429-4435-bbc4-284839fc2621","Type":"ContainerStarted","Data":"c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044"}
Sep 08 14:11:34.710356 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:34.710257    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x" event={"ID":"7bc71734-6429-4435-bbc4-284839fc2621","Type":"ContainerStarted","Data":"b9a54d3d72deb5a0f486795e4eaca6f8fb55812411103695d149c958bfd2d640"}
Sep 08 14:11:34.710396 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:34.710345    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:11:34.729112 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:34.729067    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt" podStartSLOduration=1.72904826 podStartE2EDuration="1.72904826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:11:34.725789931 +0000 UTC m=+785.220760616" watchObservedRunningTime="2026-09-08 14:11:34.72904826 +0000 UTC m=+785.224018939"
Sep 08 14:11:34.744197 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:34.744153    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x" podStartSLOduration=1.7441234479999999 podStartE2EDuration="1.744123448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:11:34.741401917 +0000 UTC m=+785.236372603" watchObservedRunningTime="2026-09-08 14:11:34.744123448 +0000 UTC m=+785.239094133"
Sep 08 14:11:45.717679 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:45.717649    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:11:45.718185 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:45.717716    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:11:55.892584 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:55.892486    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"]
Sep 08 14:11:55.896139 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:55.896120    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"
Sep 08 14:11:55.898771 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:55.898750    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:11:55.898771 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:55.898764    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:11:55.903432 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:55.903402    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"]
Sep 08 14:11:56.030527 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:56.030478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c1293ba-ba98-4b31-8578-69d3484d5f49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl\" (UID: \"6c1293ba-ba98-4b31-8578-69d3484d5f49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"
Sep 08 14:11:56.132290 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:56.132243    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c1293ba-ba98-4b31-8578-69d3484d5f49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl\" (UID: \"6c1293ba-ba98-4b31-8578-69d3484d5f49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"
Sep 08 14:11:56.133156 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:56.133124    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c1293ba-ba98-4b31-8578-69d3484d5f49-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl\" (UID: \"6c1293ba-ba98-4b31-8578-69d3484d5f49\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"
Sep 08 14:11:56.210143 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:56.210062    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"
Sep 08 14:11:56.505812 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:56.505775    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"]
Sep 08 14:11:56.507536 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:11:56.507499    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1293ba_ba98_4b31_8578_69d3484d5f49.slice/crio-39026e28d015ec6e9c6df1e86ad581f80cc74bc895e9afbe30e6100c8fb100ac WatchSource:0}: Error finding container 39026e28d015ec6e9c6df1e86ad581f80cc74bc895e9afbe30e6100c8fb100ac: Status 404 returned error can't find the container with id 39026e28d015ec6e9c6df1e86ad581f80cc74bc895e9afbe30e6100c8fb100ac
Sep 08 14:11:56.828283 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:56.828200    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl" event={"ID":"6c1293ba-ba98-4b31-8578-69d3484d5f49","Type":"ContainerStarted","Data":"39026e28d015ec6e9c6df1e86ad581f80cc74bc895e9afbe30e6100c8fb100ac"}
Sep 08 14:11:57.841948 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:57.838502    2564 generic.go:356] "Generic (PLEG): container finished" podID="6c1293ba-ba98-4b31-8578-69d3484d5f49" containerID="202006ebfc1050e04a2c3ad5df16c0fd8b4b2000e23093d747aa0d857036ab41" exitCode=0
Sep 08 14:11:57.841948 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:57.838669    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl" event={"ID":"6c1293ba-ba98-4b31-8578-69d3484d5f49","Type":"ContainerDied","Data":"202006ebfc1050e04a2c3ad5df16c0fd8b4b2000e23093d747aa0d857036ab41"}
Sep 08 14:11:58.983032 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:58.983001    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"
Sep 08 14:11:59.078737 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:59.078701    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c1293ba-ba98-4b31-8578-69d3484d5f49-maas-api-service-ca\") pod \"6c1293ba-ba98-4b31-8578-69d3484d5f49\" (UID: \"6c1293ba-ba98-4b31-8578-69d3484d5f49\") "
Sep 08 14:11:59.079165 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:59.079137    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c1293ba-ba98-4b31-8578-69d3484d5f49-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6c1293ba-ba98-4b31-8578-69d3484d5f49" (UID: "6c1293ba-ba98-4b31-8578-69d3484d5f49"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:11:59.180564 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:59.180474    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c1293ba-ba98-4b31-8578-69d3484d5f49-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:11:59.851267 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:59.851228    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"
Sep 08 14:11:59.851267 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:59.851251    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl" event={"ID":"6c1293ba-ba98-4b31-8578-69d3484d5f49","Type":"ContainerDied","Data":"39026e28d015ec6e9c6df1e86ad581f80cc74bc895e9afbe30e6100c8fb100ac"}
Sep 08 14:11:59.851477 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:11:59.851302    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39026e28d015ec6e9c6df1e86ad581f80cc74bc895e9afbe30e6100c8fb100ac"
Sep 08 14:12:01.693074 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:01.693041    2564 patch_prober.go:28] interesting pod/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 08 14:12:01.693504 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:01.693101    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx" podUID="8412890e-56a7-4b8f-b127-070e34897640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 08 14:12:01.863711 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:01.863613    2564 generic.go:356] "Generic (PLEG): container finished" podID="8412890e-56a7-4b8f-b127-070e34897640" containerID="6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e" exitCode=0
Sep 08 14:12:01.863711 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:01.863688    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx" event={"ID":"8412890e-56a7-4b8f-b127-070e34897640","Type":"ContainerDied","Data":"6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e"}
Sep 08 14:12:01.864158 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:01.864144    2564 scope.go:117] "RemoveContainer" containerID="6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e"
Sep 08 14:12:01.865357 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:01.865325    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:12:01.865447 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:01.865399    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:12:01.865447 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:01.865435    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:12:02.870977 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:02.870932    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx" event={"ID":"8412890e-56a7-4b8f-b127-070e34897640","Type":"ContainerStarted","Data":"8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a"}
Sep 08 14:12:03.130950 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.127600    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"]
Sep 08 14:12:03.135751 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.135711    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"]
Sep 08 14:12:03.136040 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.136003    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x" podUID="7bc71734-6429-4435-bbc4-284839fc2621" containerName="payload-pre-processing" containerID="cri-o://c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044" gracePeriod=30
Sep 08 14:12:03.145135 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.127851    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt" podUID="55107102-3f5b-4698-9677-e4a191f2d5a0" containerName="payload-processing" containerID="cri-o://946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e" gracePeriod=30
Sep 08 14:12:03.410904 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.410831    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"]
Sep 08 14:12:03.414136 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.414103    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-93eb17b6-5mgpl"]
Sep 08 14:12:03.461670 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.461642    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:12:03.515116 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.515083    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:12:03.545902 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.540291    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"]
Sep 08 14:12:03.655494 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.655389    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bc71734-6429-4435-bbc4-284839fc2621-plugins-config-volume\") pod \"7bc71734-6429-4435-bbc4-284839fc2621\" (UID: \"7bc71734-6429-4435-bbc4-284839fc2621\") "
Sep 08 14:12:03.655494 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.655481    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg4sw\" (UniqueName: \"kubernetes.io/projected/7bc71734-6429-4435-bbc4-284839fc2621-kube-api-access-mg4sw\") pod \"7bc71734-6429-4435-bbc4-284839fc2621\" (UID: \"7bc71734-6429-4435-bbc4-284839fc2621\") "
Sep 08 14:12:03.655737 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.655534    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55107102-3f5b-4698-9677-e4a191f2d5a0-plugins-config-volume\") pod \"55107102-3f5b-4698-9677-e4a191f2d5a0\" (UID: \"55107102-3f5b-4698-9677-e4a191f2d5a0\") "
Sep 08 14:12:03.655737 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.655642    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpl27\" (UniqueName: \"kubernetes.io/projected/55107102-3f5b-4698-9677-e4a191f2d5a0-kube-api-access-xpl27\") pod \"55107102-3f5b-4698-9677-e4a191f2d5a0\" (UID: \"55107102-3f5b-4698-9677-e4a191f2d5a0\") "
Sep 08 14:12:03.655850 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.655815    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bc71734-6429-4435-bbc4-284839fc2621-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7bc71734-6429-4435-bbc4-284839fc2621" (UID: "7bc71734-6429-4435-bbc4-284839fc2621"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:12:03.656086 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.656063    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bc71734-6429-4435-bbc4-284839fc2621-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:03.656540 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.656513    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55107102-3f5b-4698-9677-e4a191f2d5a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "55107102-3f5b-4698-9677-e4a191f2d5a0" (UID: "55107102-3f5b-4698-9677-e4a191f2d5a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:12:03.658720 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.658692    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55107102-3f5b-4698-9677-e4a191f2d5a0-kube-api-access-xpl27" (OuterVolumeSpecName: "kube-api-access-xpl27") pod "55107102-3f5b-4698-9677-e4a191f2d5a0" (UID: "55107102-3f5b-4698-9677-e4a191f2d5a0"). InnerVolumeSpecName "kube-api-access-xpl27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:12:03.658835 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.658820    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bc71734-6429-4435-bbc4-284839fc2621-kube-api-access-mg4sw" (OuterVolumeSpecName: "kube-api-access-mg4sw") pod "7bc71734-6429-4435-bbc4-284839fc2621" (UID: "7bc71734-6429-4435-bbc4-284839fc2621"). InnerVolumeSpecName "kube-api-access-mg4sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:12:03.757346 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.757305    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg4sw\" (UniqueName: \"kubernetes.io/projected/7bc71734-6429-4435-bbc4-284839fc2621-kube-api-access-mg4sw\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:03.757346 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.757339    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55107102-3f5b-4698-9677-e4a191f2d5a0-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:03.757346 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.757351    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpl27\" (UniqueName: \"kubernetes.io/projected/55107102-3f5b-4698-9677-e4a191f2d5a0-kube-api-access-xpl27\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:03.868968 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.868923    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:12:03.869247 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.869221    2564 patch_prober.go:28] interesting pod/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 08 14:12:03.869343 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.869278    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx" podUID="8412890e-56a7-4b8f-b127-070e34897640" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 08 14:12:03.877954 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.877921    2564 generic.go:356] "Generic (PLEG): container finished" podID="55107102-3f5b-4698-9677-e4a191f2d5a0" containerID="946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e" exitCode=0
Sep 08 14:12:03.878393 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.877970    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"
Sep 08 14:12:03.878393 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.877984    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt" event={"ID":"55107102-3f5b-4698-9677-e4a191f2d5a0","Type":"ContainerDied","Data":"946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e"}
Sep 08 14:12:03.878393 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.878041    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt" event={"ID":"55107102-3f5b-4698-9677-e4a191f2d5a0","Type":"ContainerDied","Data":"bf3a96ddc98511647bef525e4bcfdb062e55b27f5c6b2040d2d68694edeb58f3"}
Sep 08 14:12:03.878393 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.878062    2564 scope.go:117] "RemoveContainer" containerID="946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e"
Sep 08 14:12:03.879657 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.879637    2564 generic.go:356] "Generic (PLEG): container finished" podID="7bc71734-6429-4435-bbc4-284839fc2621" containerID="c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044" exitCode=0
Sep 08 14:12:03.879806 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.879722    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x" event={"ID":"7bc71734-6429-4435-bbc4-284839fc2621","Type":"ContainerDied","Data":"c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044"}
Sep 08 14:12:03.879806 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.879768    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x" event={"ID":"7bc71734-6429-4435-bbc4-284839fc2621","Type":"ContainerDied","Data":"b9a54d3d72deb5a0f486795e4eaca6f8fb55812411103695d149c958bfd2d640"}
Sep 08 14:12:03.879961 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.879801    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"
Sep 08 14:12:03.880057 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.879840    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx" podUID="8412890e-56a7-4b8f-b127-070e34897640" containerName="istio-proxy" containerID="cri-o://8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a" gracePeriod=30
Sep 08 14:12:03.889071 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.889054    2564 scope.go:117] "RemoveContainer" containerID="946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e"
Sep 08 14:12:03.889368 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:12:03.889340    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e\": container with ID starting with 946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e not found: ID does not exist" containerID="946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e"
Sep 08 14:12:03.889465 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.889394    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e"} err="failed to get container status \"946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e\": rpc error: code = NotFound desc = could not find container \"946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e\": container with ID starting with 946007fa644cbfbdbb7a9250180adf7bb05df5614bc76b97000effc6eb81ca2e not found: ID does not exist"
Sep 08 14:12:03.889465 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.889419    2564 scope.go:117] "RemoveContainer" containerID="c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044"
Sep 08 14:12:03.898376 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.898359    2564 scope.go:117] "RemoveContainer" containerID="c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044"
Sep 08 14:12:03.898640 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:12:03.898624    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044\": container with ID starting with c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044 not found: ID does not exist" containerID="c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044"
Sep 08 14:12:03.898683 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.898646    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044"} err="failed to get container status \"c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044\": rpc error: code = NotFound desc = could not find container \"c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044\": container with ID starting with c4520ec71765b987ed9750ba3ced3ed294468595704311c5518353418f010044 not found: ID does not exist"
Sep 08 14:12:03.930486 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.930390    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"]
Sep 08 14:12:03.934425 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.934179    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-93eb17b6-77dddb59fd-t4l9x"]
Sep 08 14:12:03.950012 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.947719    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"]
Sep 08 14:12:03.951653 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.951624    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-93eb17b6-5f98cbc6b9-pbbvt"]
Sep 08 14:12:03.959836 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.959805    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55107102-3f5b-4698-9677-e4a191f2d5a0" path="/var/lib/kubelet/pods/55107102-3f5b-4698-9677-e4a191f2d5a0/volumes"
Sep 08 14:12:03.960203 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.960184    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c1293ba-ba98-4b31-8578-69d3484d5f49" path="/var/lib/kubelet/pods/6c1293ba-ba98-4b31-8578-69d3484d5f49/volumes"
Sep 08 14:12:03.960549 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:03.960532    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bc71734-6429-4435-bbc4-284839fc2621" path="/var/lib/kubelet/pods/7bc71734-6429-4435-bbc4-284839fc2621/volumes"
Sep 08 14:12:13.138810 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.138783    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"]
Sep 08 14:12:13.139391 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.139369    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c1293ba-ba98-4b31-8578-69d3484d5f49" containerName="revoke-keys"
Sep 08 14:12:13.139481 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.139395    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c1293ba-ba98-4b31-8578-69d3484d5f49" containerName="revoke-keys"
Sep 08 14:12:13.139481 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.139421    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bc71734-6429-4435-bbc4-284839fc2621" containerName="payload-pre-processing"
Sep 08 14:12:13.139481 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.139429    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bc71734-6429-4435-bbc4-284839fc2621" containerName="payload-pre-processing"
Sep 08 14:12:13.139481 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.139441    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55107102-3f5b-4698-9677-e4a191f2d5a0" containerName="payload-processing"
Sep 08 14:12:13.139481 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.139450    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="55107102-3f5b-4698-9677-e4a191f2d5a0" containerName="payload-processing"
Sep 08 14:12:13.139728 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.139559    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bc71734-6429-4435-bbc4-284839fc2621" containerName="payload-pre-processing"
Sep 08 14:12:13.139728 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.139576    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="55107102-3f5b-4698-9677-e4a191f2d5a0" containerName="payload-processing"
Sep 08 14:12:13.139728 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.139586    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c1293ba-ba98-4b31-8578-69d3484d5f49" containerName="revoke-keys"
Sep 08 14:12:13.141689 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.141670    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"
Sep 08 14:12:13.148948 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.144681    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:12:13.148948 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.144701    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:12:13.160067 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.159173    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"]
Sep 08 14:12:13.163168 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.163146    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:12:13.247227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247197    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8412890e-56a7-4b8f-b127-070e34897640-istiod-ca-cert\") pod \"8412890e-56a7-4b8f-b127-070e34897640\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") "
Sep 08 14:12:13.247227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247228    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-istio-token\") pod \"8412890e-56a7-4b8f-b127-070e34897640\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") "
Sep 08 14:12:13.247454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247253    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gndpx\" (UniqueName: \"kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-kube-api-access-gndpx\") pod \"8412890e-56a7-4b8f-b127-070e34897640\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") "
Sep 08 14:12:13.247454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247294    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-socket\") pod \"8412890e-56a7-4b8f-b127-070e34897640\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") "
Sep 08 14:12:13.247454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247312    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8412890e-56a7-4b8f-b127-070e34897640-istio-podinfo\") pod \"8412890e-56a7-4b8f-b127-070e34897640\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") "
Sep 08 14:12:13.247454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247334    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-credential-socket\") pod \"8412890e-56a7-4b8f-b127-070e34897640\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") "
Sep 08 14:12:13.247454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247370    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-certs\") pod \"8412890e-56a7-4b8f-b127-070e34897640\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") "
Sep 08 14:12:13.247454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247398    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-data\") pod \"8412890e-56a7-4b8f-b127-070e34897640\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") "
Sep 08 14:12:13.247454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247433    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-envoy\") pod \"8412890e-56a7-4b8f-b127-070e34897640\" (UID: \"8412890e-56a7-4b8f-b127-070e34897640\") "
Sep 08 14:12:13.247817 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247661    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42dcac7c-ca88-4c4c-afc4-8198720be6dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-41383d-vq7tr\" (UID: \"42dcac7c-ca88-4c4c-afc4-8198720be6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"
Sep 08 14:12:13.247817 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247658    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8412890e-56a7-4b8f-b127-070e34897640" (UID: "8412890e-56a7-4b8f-b127-070e34897640"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:12:13.247817 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247686    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8412890e-56a7-4b8f-b127-070e34897640" (UID: "8412890e-56a7-4b8f-b127-070e34897640"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:12:13.248017 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247816    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-data" (OuterVolumeSpecName: "istio-data") pod "8412890e-56a7-4b8f-b127-070e34897640" (UID: "8412890e-56a7-4b8f-b127-070e34897640"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:12:13.248017 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247901    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:13.248017 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247922    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-workload-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:13.248017 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247969    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8412890e-56a7-4b8f-b127-070e34897640-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8412890e-56a7-4b8f-b127-070e34897640" (UID: "8412890e-56a7-4b8f-b127-070e34897640"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:12:13.248017 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.247975    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8412890e-56a7-4b8f-b127-070e34897640" (UID: "8412890e-56a7-4b8f-b127-070e34897640"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:12:13.249769 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.249729    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-istio-token" (OuterVolumeSpecName: "istio-token") pod "8412890e-56a7-4b8f-b127-070e34897640" (UID: "8412890e-56a7-4b8f-b127-070e34897640"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:12:13.249910 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.249798    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8412890e-56a7-4b8f-b127-070e34897640-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8412890e-56a7-4b8f-b127-070e34897640" (UID: "8412890e-56a7-4b8f-b127-070e34897640"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:12:13.249910 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.249815    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-kube-api-access-gndpx" (OuterVolumeSpecName: "kube-api-access-gndpx") pod "8412890e-56a7-4b8f-b127-070e34897640" (UID: "8412890e-56a7-4b8f-b127-070e34897640"). InnerVolumeSpecName "kube-api-access-gndpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:12:13.250071 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.250048    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8412890e-56a7-4b8f-b127-070e34897640" (UID: "8412890e-56a7-4b8f-b127-070e34897640"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:12:13.349381 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.349345    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42dcac7c-ca88-4c4c-afc4-8198720be6dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-41383d-vq7tr\" (UID: \"42dcac7c-ca88-4c4c-afc4-8198720be6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"
Sep 08 14:12:13.349576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.349403    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8412890e-56a7-4b8f-b127-070e34897640-istiod-ca-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:13.349576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.349413    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-istio-token\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:13.349576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.349422    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gndpx\" (UniqueName: \"kubernetes.io/projected/8412890e-56a7-4b8f-b127-070e34897640-kube-api-access-gndpx\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:13.349576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.349432    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8412890e-56a7-4b8f-b127-070e34897640-istio-podinfo\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:13.349576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.349440    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-credential-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:13.349576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.349447    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-data\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:13.349576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.349455    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8412890e-56a7-4b8f-b127-070e34897640-istio-envoy\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:13.349995 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.349973    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42dcac7c-ca88-4c4c-afc4-8198720be6dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-41383d-vq7tr\" (UID: \"42dcac7c-ca88-4c4c-afc4-8198720be6dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"
Sep 08 14:12:13.470292 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.470255    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"
Sep 08 14:12:13.762814 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.762773    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"]
Sep 08 14:12:13.769325 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:12:13.769255    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42dcac7c_ca88_4c4c_afc4_8198720be6dd.slice/crio-41de41fdccf35741f004717f989cab694ee3c067c01c0fb12664947c0021468b WatchSource:0}: Error finding container 41de41fdccf35741f004717f989cab694ee3c067c01c0fb12664947c0021468b: Status 404 returned error can't find the container with id 41de41fdccf35741f004717f989cab694ee3c067c01c0fb12664947c0021468b
Sep 08 14:12:13.942188 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.942143    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr" event={"ID":"42dcac7c-ca88-4c4c-afc4-8198720be6dd","Type":"ContainerStarted","Data":"41de41fdccf35741f004717f989cab694ee3c067c01c0fb12664947c0021468b"}
Sep 08 14:12:13.943806 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.943786    2564 generic.go:356] "Generic (PLEG): container finished" podID="8412890e-56a7-4b8f-b127-070e34897640" containerID="8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a" exitCode=0
Sep 08 14:12:13.943934 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.943833    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx" event={"ID":"8412890e-56a7-4b8f-b127-070e34897640","Type":"ContainerDied","Data":"8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a"}
Sep 08 14:12:13.943934 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.943839    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"
Sep 08 14:12:13.943934 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.943881    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx" event={"ID":"8412890e-56a7-4b8f-b127-070e34897640","Type":"ContainerDied","Data":"86121a8a4c09a8463cc23208379fe2e935e6cdd5ad1adcbba54e040acbfeb6c6"}
Sep 08 14:12:13.943934 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.943907    2564 scope.go:117] "RemoveContainer" containerID="8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a"
Sep 08 14:12:13.970715 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.970685    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"]
Sep 08 14:12:13.980141 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:13.980114    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-93eb17b6-openshift-default-7548cf9fd4-kvgpx"]
Sep 08 14:12:14.026396 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:14.026246    2564 scope.go:117] "RemoveContainer" containerID="6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e"
Sep 08 14:12:14.035294 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:14.035267    2564 scope.go:117] "RemoveContainer" containerID="8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a"
Sep 08 14:12:14.035643 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:12:14.035616    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a\": container with ID starting with 8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a not found: ID does not exist" containerID="8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a"
Sep 08 14:12:14.035735 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:14.035655    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a"} err="failed to get container status \"8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a\": rpc error: code = NotFound desc = could not find container \"8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a\": container with ID starting with 8196072f7a0257bab5241cb3d64223d53f490e67918bb84f916608401750cc1a not found: ID does not exist"
Sep 08 14:12:14.035735 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:14.035683    2564 scope.go:117] "RemoveContainer" containerID="6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e"
Sep 08 14:12:14.036058 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:12:14.036039    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e\": container with ID starting with 6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e not found: ID does not exist" containerID="6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e"
Sep 08 14:12:14.036141 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:14.036064    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e"} err="failed to get container status \"6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e\": rpc error: code = NotFound desc = could not find container \"6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e\": container with ID starting with 6973f5fb80208332a1769b0474fdceb7a9b3eaee1d753250286039e6e70bb44e not found: ID does not exist"
Sep 08 14:12:14.951785 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:14.951750    2564 generic.go:356] "Generic (PLEG): container finished" podID="42dcac7c-ca88-4c4c-afc4-8198720be6dd" containerID="b9f5ff83ba00140ec574498ea594cb0afe296235d37b455b66428c72d26d4841" exitCode=0
Sep 08 14:12:14.952191 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:14.951811    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr" event={"ID":"42dcac7c-ca88-4c4c-afc4-8198720be6dd","Type":"ContainerDied","Data":"b9f5ff83ba00140ec574498ea594cb0afe296235d37b455b66428c72d26d4841"}
Sep 08 14:12:15.959250 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:15.959225    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8412890e-56a7-4b8f-b127-070e34897640" path="/var/lib/kubelet/pods/8412890e-56a7-4b8f-b127-070e34897640/volumes"
Sep 08 14:12:16.092410 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:16.092386    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"
Sep 08 14:12:16.175835 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:16.175804    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42dcac7c-ca88-4c4c-afc4-8198720be6dd-maas-api-service-ca\") pod \"42dcac7c-ca88-4c4c-afc4-8198720be6dd\" (UID: \"42dcac7c-ca88-4c4c-afc4-8198720be6dd\") "
Sep 08 14:12:16.176184 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:16.176162    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42dcac7c-ca88-4c4c-afc4-8198720be6dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "42dcac7c-ca88-4c4c-afc4-8198720be6dd" (UID: "42dcac7c-ca88-4c4c-afc4-8198720be6dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:12:16.276850 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:16.276809    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42dcac7c-ca88-4c4c-afc4-8198720be6dd-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:16.964118 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:16.964087    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"
Sep 08 14:12:16.964535 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:16.964124    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr" event={"ID":"42dcac7c-ca88-4c4c-afc4-8198720be6dd","Type":"ContainerDied","Data":"41de41fdccf35741f004717f989cab694ee3c067c01c0fb12664947c0021468b"}
Sep 08 14:12:16.964535 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:16.964169    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41de41fdccf35741f004717f989cab694ee3c067c01c0fb12664947c0021468b"
Sep 08 14:12:23.698376 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:23.698343    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"]
Sep 08 14:12:23.698818 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:23.698587    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4" podUID="3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98" containerName="payload-pre-processing" containerID="cri-o://0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd" gracePeriod=30
Sep 08 14:12:23.879704 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:23.879670    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"]
Sep 08 14:12:23.886753 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:23.885835    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-41383d-vq7tr"]
Sep 08 14:12:23.955216 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:23.955152    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:12:23.959584 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:23.959543    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42dcac7c-ca88-4c4c-afc4-8198720be6dd" path="/var/lib/kubelet/pods/42dcac7c-ca88-4c4c-afc4-8198720be6dd/volumes"
Sep 08 14:12:24.001930 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.001893    2564 generic.go:356] "Generic (PLEG): container finished" podID="3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98" containerID="0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd" exitCode=0
Sep 08 14:12:24.002138 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.001954    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"
Sep 08 14:12:24.002138 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.001991    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4" event={"ID":"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98","Type":"ContainerDied","Data":"0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd"}
Sep 08 14:12:24.002138 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.002048    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4" event={"ID":"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98","Type":"ContainerDied","Data":"38c69ef970c5270502e17a05f72bfac273b09d3b1f25e98f5b932317ea79a0e6"}
Sep 08 14:12:24.002138 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.002075    2564 scope.go:117] "RemoveContainer" containerID="0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd"
Sep 08 14:12:24.011666 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.011645    2564 scope.go:117] "RemoveContainer" containerID="0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd"
Sep 08 14:12:24.012016 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:12:24.011996    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd\": container with ID starting with 0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd not found: ID does not exist" containerID="0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd"
Sep 08 14:12:24.012101 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.012024    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd"} err="failed to get container status \"0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd\": rpc error: code = NotFound desc = could not find container \"0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd\": container with ID starting with 0e97c49c4c001d0e6deedfc3cd1e46fbe90440ddeab211521a44f85a2033ecfd not found: ID does not exist"
Sep 08 14:12:24.055427 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.055389    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v58q9\" (UniqueName: \"kubernetes.io/projected/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-kube-api-access-v58q9\") pod \"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98\" (UID: \"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98\") "
Sep 08 14:12:24.055617 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.055476    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-plugins-config-volume\") pod \"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98\" (UID: \"3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98\") "
Sep 08 14:12:24.055834 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.055805    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98" (UID: "3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:12:24.057706 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.057685    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-kube-api-access-v58q9" (OuterVolumeSpecName: "kube-api-access-v58q9") pod "3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98" (UID: "3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98"). InnerVolumeSpecName "kube-api-access-v58q9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:12:24.156449 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.156406    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:24.156449 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.156443    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v58q9\" (UniqueName: \"kubernetes.io/projected/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98-kube-api-access-v58q9\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:12:24.325422 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.325381    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"]
Sep 08 14:12:24.331530 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:24.331493    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-41383d-6d7594ccc4-qgxd4"]
Sep 08 14:12:25.958527 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:25.958494    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98" path="/var/lib/kubelet/pods/3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98/volumes"
Sep 08 14:12:38.899323 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899288    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"]
Sep 08 14:12:38.899730 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899652    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8412890e-56a7-4b8f-b127-070e34897640" containerName="istio-proxy"
Sep 08 14:12:38.899730 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899663    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8412890e-56a7-4b8f-b127-070e34897640" containerName="istio-proxy"
Sep 08 14:12:38.899730 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899676    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42dcac7c-ca88-4c4c-afc4-8198720be6dd" containerName="revoke-keys"
Sep 08 14:12:38.899730 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899682    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="42dcac7c-ca88-4c4c-afc4-8198720be6dd" containerName="revoke-keys"
Sep 08 14:12:38.899730 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899697    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98" containerName="payload-pre-processing"
Sep 08 14:12:38.899730 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899702    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98" containerName="payload-pre-processing"
Sep 08 14:12:38.899730 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899710    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8412890e-56a7-4b8f-b127-070e34897640" containerName="istio-proxy"
Sep 08 14:12:38.899730 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899715    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8412890e-56a7-4b8f-b127-070e34897640" containerName="istio-proxy"
Sep 08 14:12:38.900034 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899771    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8412890e-56a7-4b8f-b127-070e34897640" containerName="istio-proxy"
Sep 08 14:12:38.900034 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899781    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8412890e-56a7-4b8f-b127-070e34897640" containerName="istio-proxy"
Sep 08 14:12:38.900034 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899788    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ce0c48f-1a10-4edc-8f9c-db8b59cbfa98" containerName="payload-pre-processing"
Sep 08 14:12:38.900034 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.899794    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="42dcac7c-ca88-4c4c-afc4-8198720be6dd" containerName="revoke-keys"
Sep 08 14:12:38.904506 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.904488    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:38.907134 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.907112    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-ce93b475-kserve-self-signed-certs\""
Sep 08 14:12:38.913951 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:38.913930    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"]
Sep 08 14:12:39.100194 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.100162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-dshm\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.100361 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.100212    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-model-cache\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.100361 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.100310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tls-certs\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.100361 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.100340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tmp-dir\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.100472 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.100380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6kjq\" (UniqueName: \"kubernetes.io/projected/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-kube-api-access-v6kjq\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.100472 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.100411    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-home\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.201906 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.201804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6kjq\" (UniqueName: \"kubernetes.io/projected/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-kube-api-access-v6kjq\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.201906 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.201837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-home\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.202148 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.201919    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-dshm\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.202148 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.201974    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-model-cache\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.202148 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.202037    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tls-certs\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.202148 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.202067    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tmp-dir\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.202452 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.202405    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-model-cache\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.202578 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.202417    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-home\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.202578 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.202510    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tmp-dir\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.204353 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.204322    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-dshm\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.204753 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.204732    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tls-certs\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.210811 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.210783    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6kjq\" (UniqueName: \"kubernetes.io/projected/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-kube-api-access-v6kjq\") pod \"e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") " pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.213752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.213731    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:39.488205 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:39.488179    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"]
Sep 08 14:12:39.490201 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:12:39.490167    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabcfc13a_bbc5_4de4_a7ac_e61c348361e0.slice/crio-25adb9bc48e2cfb392cbb59a73d40889eb946f79fb9a09584d2eae3da6e308e8 WatchSource:0}: Error finding container 25adb9bc48e2cfb392cbb59a73d40889eb946f79fb9a09584d2eae3da6e308e8: Status 404 returned error can't find the container with id 25adb9bc48e2cfb392cbb59a73d40889eb946f79fb9a09584d2eae3da6e308e8
Sep 08 14:12:40.089308 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:40.089275    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2" event={"ID":"abcfc13a-bbc5-4de4-a7ac-e61c348361e0","Type":"ContainerStarted","Data":"924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da"}
Sep 08 14:12:40.089308 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:40.089317    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2" event={"ID":"abcfc13a-bbc5-4de4-a7ac-e61c348361e0","Type":"ContainerStarted","Data":"25adb9bc48e2cfb392cbb59a73d40889eb946f79fb9a09584d2eae3da6e308e8"}
Sep 08 14:12:40.109249 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:40.109197    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2" podStartSLOduration=2.109180354 podStartE2EDuration="2.109180354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:12:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:12:40.107435638 +0000 UTC m=+850.602406321" watchObservedRunningTime="2026-09-08 14:12:40.109180354 +0000 UTC m=+850.604151038"
Sep 08 14:12:40.214495 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:40.214461    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:49.214305 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:49.214272    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:49.227062 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:49.227032    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:12:50.152315 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:12:50.152286    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:13:05.788762 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:05.788729    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"]
Sep 08 14:13:08.009561 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.009527    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"]
Sep 08 14:13:08.013302 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.013285    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.015837 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.015811    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-3b812d-openshift-default-dockercfg-xvtfp\""
Sep 08 14:13:08.027765 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.027740    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"]
Sep 08 14:13:08.179374 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.179342    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f1c296e-5446-4953-a145-419f99fa71b8-istiod-ca-cert\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.179548 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.179379    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-socket\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.179548 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.179421    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-certs\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.179548 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.179486    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-envoy\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.179548 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.179540    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-data\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.179701 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.179577    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpjrs\" (UniqueName: \"kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-kube-api-access-jpjrs\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.179701 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.179598    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-credential-socket\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.179701 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.179611    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-istio-token\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.179800 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.179713    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f1c296e-5446-4953-a145-419f99fa71b8-istio-podinfo\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.281322 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.281217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-socket\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.281322 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.281306    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-certs\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.281606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.281353    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-envoy\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.281606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.281397    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-data\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.281606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.281433    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpjrs\" (UniqueName: \"kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-kube-api-access-jpjrs\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.281606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.281461    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-credential-socket\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.281606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.281487    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-istio-token\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.281606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.281582    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f1c296e-5446-4953-a145-419f99fa71b8-istio-podinfo\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.281965 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.281621    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f1c296e-5446-4953-a145-419f99fa71b8-istiod-ca-cert\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.282178 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.282134    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-socket\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.282472 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.282163    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-credential-socket\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.283942 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.283912    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f1c296e-5446-4953-a145-419f99fa71b8-istiod-ca-cert\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.284316 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.284291    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-data\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.284530 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.284506    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-certs\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.284876 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.284837    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-envoy\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.286771 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.286749    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f1c296e-5446-4953-a145-419f99fa71b8-istio-podinfo\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.302292 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.302263    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-istio-token\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.302521 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.302496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpjrs\" (UniqueName: \"kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-kube-api-access-jpjrs\") pod \"e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") " pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.324536 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.324506    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:08.619667 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.619631    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"]
Sep 08 14:13:08.620913 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:13:08.620876    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f1c296e_5446_4953_a145_419f99fa71b8.slice/crio-fad6332a611a8b06e7b1122776c8f23cd7fcb43f09cda038c32f2636a9c15ed3 WatchSource:0}: Error finding container fad6332a611a8b06e7b1122776c8f23cd7fcb43f09cda038c32f2636a9c15ed3: Status 404 returned error can't find the container with id fad6332a611a8b06e7b1122776c8f23cd7fcb43f09cda038c32f2636a9c15ed3
Sep 08 14:13:08.623083 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.623048    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:13:08.623181 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.623114    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:13:08.623181 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:08.623141    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:13:09.248501 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:09.248464    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" event={"ID":"9f1c296e-5446-4953-a145-419f99fa71b8","Type":"ContainerStarted","Data":"1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b"}
Sep 08 14:13:09.249045 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:09.248528    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" event={"ID":"9f1c296e-5446-4953-a145-419f99fa71b8","Type":"ContainerStarted","Data":"fad6332a611a8b06e7b1122776c8f23cd7fcb43f09cda038c32f2636a9c15ed3"}
Sep 08 14:13:09.270959 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:09.270904    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" podStartSLOduration=2.270884365 podStartE2EDuration="2.270884365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:13:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:13:09.268987761 +0000 UTC m=+879.763958446" watchObservedRunningTime="2026-09-08 14:13:09.270884365 +0000 UTC m=+879.765855040"
Sep 08 14:13:09.325065 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:09.325021    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:09.330279 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:09.330253    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:10.254115 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:10.254088    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:10.255182 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:10.255163    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:20.809432 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:20.809323    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2" podUID="abcfc13a-bbc5-4de4-a7ac-e61c348361e0" containerName="main" containerID="cri-o://924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da" gracePeriod=45
Sep 08 14:13:20.876268 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:13:20.876234    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabcfc13a_bbc5_4de4_a7ac_e61c348361e0.slice/crio-924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabcfc13a_bbc5_4de4_a7ac_e61c348361e0.slice/crio-conmon-924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da.scope\": RecentStats: unable to find data in memory cache]"
Sep 08 14:13:21.063165 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.063104    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:13:21.105609 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.105579    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-home\") pod \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") "
Sep 08 14:13:21.105781 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.105658    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-model-cache\") pod \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") "
Sep 08 14:13:21.105781 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.105683    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-dshm\") pod \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") "
Sep 08 14:13:21.105781 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.105769    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6kjq\" (UniqueName: \"kubernetes.io/projected/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-kube-api-access-v6kjq\") pod \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") "
Sep 08 14:13:21.105965 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.105798    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tmp-dir\") pod \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") "
Sep 08 14:13:21.105965 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.105878    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tls-certs\") pod \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\" (UID: \"abcfc13a-bbc5-4de4-a7ac-e61c348361e0\") "
Sep 08 14:13:21.105965 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.105900    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-home" (OuterVolumeSpecName: "home") pod "abcfc13a-bbc5-4de4-a7ac-e61c348361e0" (UID: "abcfc13a-bbc5-4de4-a7ac-e61c348361e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:13:21.105965 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.105915    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "abcfc13a-bbc5-4de4-a7ac-e61c348361e0" (UID: "abcfc13a-bbc5-4de4-a7ac-e61c348361e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:13:21.106156 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.106135    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "abcfc13a-bbc5-4de4-a7ac-e61c348361e0" (UID: "abcfc13a-bbc5-4de4-a7ac-e61c348361e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:13:21.106218 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.106203    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-model-cache\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:13:21.106267 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.106222    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-home\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:13:21.108217 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.108194    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "abcfc13a-bbc5-4de4-a7ac-e61c348361e0" (UID: "abcfc13a-bbc5-4de4-a7ac-e61c348361e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:13:21.108500 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.108481    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-dshm" (OuterVolumeSpecName: "dshm") pod "abcfc13a-bbc5-4de4-a7ac-e61c348361e0" (UID: "abcfc13a-bbc5-4de4-a7ac-e61c348361e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:13:21.108561 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.108514    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-kube-api-access-v6kjq" (OuterVolumeSpecName: "kube-api-access-v6kjq") pod "abcfc13a-bbc5-4de4-a7ac-e61c348361e0" (UID: "abcfc13a-bbc5-4de4-a7ac-e61c348361e0"). InnerVolumeSpecName "kube-api-access-v6kjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:13:21.207191 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.207159    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tls-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:13:21.207191 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.207186    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-dshm\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:13:21.207191 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.207195    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6kjq\" (UniqueName: \"kubernetes.io/projected/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-kube-api-access-v6kjq\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:13:21.207415 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.207203    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abcfc13a-bbc5-4de4-a7ac-e61c348361e0-tmp-dir\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:13:21.314918 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.314803    2564 generic.go:356] "Generic (PLEG): container finished" podID="abcfc13a-bbc5-4de4-a7ac-e61c348361e0" containerID="924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da" exitCode=0
Sep 08 14:13:21.314918 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.314836    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"
Sep 08 14:13:21.315102 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.314897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2" event={"ID":"abcfc13a-bbc5-4de4-a7ac-e61c348361e0","Type":"ContainerDied","Data":"924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da"}
Sep 08 14:13:21.315102 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.315033    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2" event={"ID":"abcfc13a-bbc5-4de4-a7ac-e61c348361e0","Type":"ContainerDied","Data":"25adb9bc48e2cfb392cbb59a73d40889eb946f79fb9a09584d2eae3da6e308e8"}
Sep 08 14:13:21.315102 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.315055    2564 scope.go:117] "RemoveContainer" containerID="924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da"
Sep 08 14:13:21.324492 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.324476    2564 scope.go:117] "RemoveContainer" containerID="924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da"
Sep 08 14:13:21.324749 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:13:21.324732    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da\": container with ID starting with 924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da not found: ID does not exist" containerID="924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da"
Sep 08 14:13:21.324798 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.324762    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da"} err="failed to get container status \"924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da\": rpc error: code = NotFound desc = could not find container \"924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da\": container with ID starting with 924c338eb1835260fe2542b0742421fe573534ccbf92172e1423d180ef36b1da not found: ID does not exist"
Sep 08 14:13:21.342684 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.342660    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"]
Sep 08 14:13:21.352694 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.352672    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-ce93b475-kserve-8679f68474-4gqb2"]
Sep 08 14:13:21.958486 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:21.958454    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abcfc13a-bbc5-4de4-a7ac-e61c348361e0" path="/var/lib/kubelet/pods/abcfc13a-bbc5-4de4-a7ac-e61c348361e0/volumes"
Sep 08 14:13:33.832525 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:33.832438    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:13:33.837496 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:33.837472    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:13:34.401212 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:34.401169    2564 generic.go:356] "Generic (PLEG): container finished" podID="9f1c296e-5446-4953-a145-419f99fa71b8" containerID="1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b" exitCode=0
Sep 08 14:13:34.401383 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:34.401243    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" event={"ID":"9f1c296e-5446-4953-a145-419f99fa71b8","Type":"ContainerDied","Data":"1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b"}
Sep 08 14:13:34.401743 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:34.401725    2564 scope.go:117] "RemoveContainer" containerID="1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b"
Sep 08 14:13:34.402908 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:34.402852    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:13:34.403020 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:34.402958    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:13:34.403083 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:34.403045    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:13:35.408756 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:35.408723    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" event={"ID":"9f1c296e-5446-4953-a145-419f99fa71b8","Type":"ContainerStarted","Data":"6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df"}
Sep 08 14:13:36.327056 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:36.326703    2564 patch_prober.go:28] interesting pod/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 08 14:13:36.327056 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:36.326779    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 08 14:13:36.327409 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:36.327390    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:37.325981 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:37.325948    2564 patch_prober.go:28] interesting pod/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 08 14:13:37.326698 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:37.326664    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 08 14:13:38.325293 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:38.325247    2564 patch_prober.go:28] interesting pod/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 08 14:13:38.325509 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:38.325331    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 08 14:13:39.325298 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:39.324836    2564 patch_prober.go:28] interesting pod/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 08 14:13:39.325298 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:39.324927    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 08 14:13:40.254185 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:40.254148    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:40.325222 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:40.325177    2564 patch_prober.go:28] interesting pod/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 08 14:13:40.325427 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:40.325247    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 08 14:13:41.329199 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:41.329167    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:41.330299 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:41.330279    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:13:43.658522 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.658489    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"]
Sep 08 14:13:43.658921 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.658906    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abcfc13a-bbc5-4de4-a7ac-e61c348361e0" containerName="main"
Sep 08 14:13:43.658971 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.658922    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="abcfc13a-bbc5-4de4-a7ac-e61c348361e0" containerName="main"
Sep 08 14:13:43.659017 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.658985    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="abcfc13a-bbc5-4de4-a7ac-e61c348361e0" containerName="main"
Sep 08 14:13:43.662287 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.662271    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.670398 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.670373    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a47a1a97-openshift-default-dockercfg-jdkxd\""
Sep 08 14:13:43.681225 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.681201    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"]
Sep 08 14:13:43.750522 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.750480    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-credential-socket\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.750692 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.750541    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-socket\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.750692 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.750641    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-envoy\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.750692 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.750676    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-token\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.750877 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.750707    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a35854c5-2d99-4a16-911d-aa17eda1e21d-istiod-ca-cert\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.750877 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.750732    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-podinfo\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.750976 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.750883    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-certs\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.750976 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.750961    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g7n6\" (UniqueName: \"kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-kube-api-access-4g7n6\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.751064 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.751016    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-data\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.851987 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.851948    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-envoy\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.851987 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.851986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-token\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.852203 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.852003    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a35854c5-2d99-4a16-911d-aa17eda1e21d-istiod-ca-cert\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.852203 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.852138    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-podinfo\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.852297 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.852252    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-certs\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.852343 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.852309    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g7n6\" (UniqueName: \"kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-kube-api-access-4g7n6\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.852392 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.852370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-data\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.852449 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.852423    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-credential-socket\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.852498 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.852466    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-socket\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.852741 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.852714    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-data\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.852917 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.852893    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a35854c5-2d99-4a16-911d-aa17eda1e21d-istiod-ca-cert\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.853017 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.853002    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-certs\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.853168 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.853153    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-credential-socket\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.853241 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.853219    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-socket\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.854754 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.854730    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-envoy\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.854833 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.854780    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-podinfo\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.865420 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.865388    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-token\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.867025 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.867003    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g7n6\" (UniqueName: \"kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-kube-api-access-4g7n6\") pod \"e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") " pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:43.972585 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:43.972550    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:44.257333 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:44.257310    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"]
Sep 08 14:13:44.259456 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:13:44.259433    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda35854c5_2d99_4a16_911d_aa17eda1e21d.slice/crio-fb14b62e6f3ac92dce0452a19b94ead5411f9e46ba9b1aff0bbe0944399d693a WatchSource:0}: Error finding container fb14b62e6f3ac92dce0452a19b94ead5411f9e46ba9b1aff0bbe0944399d693a: Status 404 returned error can't find the container with id fb14b62e6f3ac92dce0452a19b94ead5411f9e46ba9b1aff0bbe0944399d693a
Sep 08 14:13:44.261674 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:44.261642    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:13:44.261728 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:44.261708    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:13:44.261765 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:44.261736    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:13:44.465387 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:44.465356    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" event={"ID":"a35854c5-2d99-4a16-911d-aa17eda1e21d","Type":"ContainerStarted","Data":"0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b"}
Sep 08 14:13:44.465583 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:44.465405    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" event={"ID":"a35854c5-2d99-4a16-911d-aa17eda1e21d","Type":"ContainerStarted","Data":"fb14b62e6f3ac92dce0452a19b94ead5411f9e46ba9b1aff0bbe0944399d693a"}
Sep 08 14:13:44.522506 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:44.522392    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" podStartSLOduration=1.522376449 podStartE2EDuration="1.522376449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:13:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:13:44.517535704 +0000 UTC m=+915.012506387" watchObservedRunningTime="2026-09-08 14:13:44.522376449 +0000 UTC m=+915.017347133"
Sep 08 14:13:44.973677 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:44.973642    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:45.978837 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:45.978809    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:46.476417 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:46.476384    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:46.477566 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:46.477536    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:13:48.560422 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.560385    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"]
Sep 08 14:13:48.564345 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.564323    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:13:48.568336 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.568320    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-a47a1a97-dockercfg-n6wmh\""
Sep 08 14:13:48.568615 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.568596    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-a47a1a97\""
Sep 08 14:13:48.577831 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.577810    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"]
Sep 08 14:13:48.698915 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.698878    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85e5b001-4b4b-4392-a052-59f701c6a5bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7\" (UID: \"85e5b001-4b4b-4392-a052-59f701c6a5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:13:48.699285 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.699252    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkmwq\" (UniqueName: \"kubernetes.io/projected/85e5b001-4b4b-4392-a052-59f701c6a5bd-kube-api-access-bkmwq\") pod \"payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7\" (UID: \"85e5b001-4b4b-4392-a052-59f701c6a5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:13:48.800392 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.800357    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkmwq\" (UniqueName: \"kubernetes.io/projected/85e5b001-4b4b-4392-a052-59f701c6a5bd-kube-api-access-bkmwq\") pod \"payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7\" (UID: \"85e5b001-4b4b-4392-a052-59f701c6a5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:13:48.800569 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.800447    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85e5b001-4b4b-4392-a052-59f701c6a5bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7\" (UID: \"85e5b001-4b4b-4392-a052-59f701c6a5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:13:48.801130 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.801109    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85e5b001-4b4b-4392-a052-59f701c6a5bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7\" (UID: \"85e5b001-4b4b-4392-a052-59f701c6a5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:13:48.812260 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.812194    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkmwq\" (UniqueName: \"kubernetes.io/projected/85e5b001-4b4b-4392-a052-59f701c6a5bd-kube-api-access-bkmwq\") pod \"payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7\" (UID: \"85e5b001-4b4b-4392-a052-59f701c6a5bd\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:13:48.875172 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:48.875132    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:13:49.150788 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:49.150762    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"]
Sep 08 14:13:49.153098 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:13:49.153062    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e5b001_4b4b_4392_a052_59f701c6a5bd.slice/crio-4791974c534886c6a69b559c254d0fed99c9fa587b5524f01abb59c1cc235259 WatchSource:0}: Error finding container 4791974c534886c6a69b559c254d0fed99c9fa587b5524f01abb59c1cc235259: Status 404 returned error can't find the container with id 4791974c534886c6a69b559c254d0fed99c9fa587b5524f01abb59c1cc235259
Sep 08 14:13:49.492822 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:49.492794    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7" event={"ID":"85e5b001-4b4b-4392-a052-59f701c6a5bd","Type":"ContainerStarted","Data":"fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f"}
Sep 08 14:13:49.493025 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:49.492843    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7" event={"ID":"85e5b001-4b4b-4392-a052-59f701c6a5bd","Type":"ContainerStarted","Data":"4791974c534886c6a69b559c254d0fed99c9fa587b5524f01abb59c1cc235259"}
Sep 08 14:13:49.493025 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:49.492934    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:13:49.516229 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:13:49.516174    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7" podStartSLOduration=1.5161558579999999 podStartE2EDuration="1.516155858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:13:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:13:49.512552893 +0000 UTC m=+920.007523578" watchObservedRunningTime="2026-09-08 14:13:49.516155858 +0000 UTC m=+920.011126543"
Sep 08 14:14:00.500575 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:00.500539    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:14:02.414917 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:02.414834    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"]
Sep 08 14:14:02.419024 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:02.419004    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"
Sep 08 14:14:02.421731 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:02.421708    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:14:02.422226 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:02.422206    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:14:02.424444 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:02.424420    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"]
Sep 08 14:14:02.538503 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:02.538472    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9497f8-7553-46ef-9ce5-fa6ee8eacb53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4\" (UID: \"4f9497f8-7553-46ef-9ce5-fa6ee8eacb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"
Sep 08 14:14:02.639265 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:02.639231    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9497f8-7553-46ef-9ce5-fa6ee8eacb53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4\" (UID: \"4f9497f8-7553-46ef-9ce5-fa6ee8eacb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"
Sep 08 14:14:02.639804 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:02.639784    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9497f8-7553-46ef-9ce5-fa6ee8eacb53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4\" (UID: \"4f9497f8-7553-46ef-9ce5-fa6ee8eacb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"
Sep 08 14:14:02.734524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:02.734481    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"
Sep 08 14:14:03.015034 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:03.015006    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"]
Sep 08 14:14:03.018026 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:14:03.017996    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f9497f8_7553_46ef_9ce5_fa6ee8eacb53.slice/crio-c2d63f1a7dc8b6b62cd93dba4d9c4cc9032be31621dac6b48494a46c8d30f8a2 WatchSource:0}: Error finding container c2d63f1a7dc8b6b62cd93dba4d9c4cc9032be31621dac6b48494a46c8d30f8a2: Status 404 returned error can't find the container with id c2d63f1a7dc8b6b62cd93dba4d9c4cc9032be31621dac6b48494a46c8d30f8a2
Sep 08 14:14:03.582361 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:03.582315    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4" event={"ID":"4f9497f8-7553-46ef-9ce5-fa6ee8eacb53","Type":"ContainerStarted","Data":"f968fb475199ebccb65c35b054b660a974801308c55adb001a4400be46383691"}
Sep 08 14:14:03.582738 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:03.582386    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4" event={"ID":"4f9497f8-7553-46ef-9ce5-fa6ee8eacb53","Type":"ContainerStarted","Data":"c2d63f1a7dc8b6b62cd93dba4d9c4cc9032be31621dac6b48494a46c8d30f8a2"}
Sep 08 14:14:03.600713 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:03.600646    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4" podStartSLOduration=1.600625395 podStartE2EDuration="1.600625395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:14:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:14:03.598180424 +0000 UTC m=+934.093151111" watchObservedRunningTime="2026-09-08 14:14:03.600625395 +0000 UTC m=+934.095596081"
Sep 08 14:14:04.590166 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:04.590127    2564 generic.go:356] "Generic (PLEG): container finished" podID="4f9497f8-7553-46ef-9ce5-fa6ee8eacb53" containerID="f968fb475199ebccb65c35b054b660a974801308c55adb001a4400be46383691" exitCode=0
Sep 08 14:14:04.590593 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:04.590218    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4" event={"ID":"4f9497f8-7553-46ef-9ce5-fa6ee8eacb53","Type":"ContainerDied","Data":"f968fb475199ebccb65c35b054b660a974801308c55adb001a4400be46383691"}
Sep 08 14:14:05.598012 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:05.597975    2564 generic.go:356] "Generic (PLEG): container finished" podID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerID="0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b" exitCode=0
Sep 08 14:14:05.598466 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:05.598054    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" event={"ID":"a35854c5-2d99-4a16-911d-aa17eda1e21d","Type":"ContainerDied","Data":"0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b"}
Sep 08 14:14:05.598929 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:05.598539    2564 scope.go:117] "RemoveContainer" containerID="0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b"
Sep 08 14:14:05.599685 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:05.599649    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:14:05.599812 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:05.599738    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:14:05.599812 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:05.599787    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:14:05.755627 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:05.755603    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"
Sep 08 14:14:05.874403 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:05.874325    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9497f8-7553-46ef-9ce5-fa6ee8eacb53-maas-api-service-ca\") pod \"4f9497f8-7553-46ef-9ce5-fa6ee8eacb53\" (UID: \"4f9497f8-7553-46ef-9ce5-fa6ee8eacb53\") "
Sep 08 14:14:05.874738 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:05.874716    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f9497f8-7553-46ef-9ce5-fa6ee8eacb53-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f9497f8-7553-46ef-9ce5-fa6ee8eacb53" (UID: "4f9497f8-7553-46ef-9ce5-fa6ee8eacb53"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:14:05.874830 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:05.874814    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9497f8-7553-46ef-9ce5-fa6ee8eacb53-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:06.606133 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:06.606101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" event={"ID":"a35854c5-2d99-4a16-911d-aa17eda1e21d","Type":"ContainerStarted","Data":"66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87"}
Sep 08 14:14:06.607424 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:06.607405    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"
Sep 08 14:14:06.607424 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:06.607410    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4" event={"ID":"4f9497f8-7553-46ef-9ce5-fa6ee8eacb53","Type":"ContainerDied","Data":"c2d63f1a7dc8b6b62cd93dba4d9c4cc9032be31621dac6b48494a46c8d30f8a2"}
Sep 08 14:14:06.607619 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:06.607453    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2d63f1a7dc8b6b62cd93dba4d9c4cc9032be31621dac6b48494a46c8d30f8a2"
Sep 08 14:14:06.973696 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:06.973659    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:14:06.973903 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:06.973847    2564 patch_prober.go:28] interesting pod/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 08 14:14:06.973980 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:06.973926    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 08 14:14:07.973854 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:07.973823    2564 patch_prober.go:28] interesting pod/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 08 14:14:07.974249 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:07.973895    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 08 14:14:08.973195 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:08.973160    2564 patch_prober.go:28] interesting pod/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 08 14:14:08.973400 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:08.973319    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 08 14:14:09.973196 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:09.973159    2564 patch_prober.go:28] interesting pod/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 08 14:14:09.973635 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:09.973216    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 08 14:14:10.973176 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:10.973136    2564 patch_prober.go:28] interesting pod/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 08 14:14:10.973371 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:10.973207    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 08 14:14:11.973593 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:11.973558    2564 patch_prober.go:28] interesting pod/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 08 14:14:11.974010 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:11.973616    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 08 14:14:13.010281 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:13.010251    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:14:13.010778 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:13.010578    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:14:13.011439 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:13.011410    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:14:13.937440 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:13.937402    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"]
Sep 08 14:14:13.944526 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:13.944490    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-3b812d-rg8x4"]
Sep 08 14:14:13.962398 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:13.962369    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f9497f8-7553-46ef-9ce5-fa6ee8eacb53" path="/var/lib/kubelet/pods/4f9497f8-7553-46ef-9ce5-fa6ee8eacb53/volumes"
Sep 08 14:14:21.171350 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.171310    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"]
Sep 08 14:14:21.171939 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.171920    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f9497f8-7553-46ef-9ce5-fa6ee8eacb53" containerName="revoke-keys"
Sep 08 14:14:21.172048 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.171942    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f9497f8-7553-46ef-9ce5-fa6ee8eacb53" containerName="revoke-keys"
Sep 08 14:14:21.172104 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.172055    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f9497f8-7553-46ef-9ce5-fa6ee8eacb53" containerName="revoke-keys"
Sep 08 14:14:21.174719 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.174696    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"
Sep 08 14:14:21.179340 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.179322    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:14:21.179451 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.179350    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:14:21.197436 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.197400    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"]
Sep 08 14:14:21.251193 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.251164    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b78b6ba-f511-40bf-88d5-e15d90d034b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66\" (UID: \"9b78b6ba-f511-40bf-88d5-e15d90d034b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"
Sep 08 14:14:21.352161 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.352127    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b78b6ba-f511-40bf-88d5-e15d90d034b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66\" (UID: \"9b78b6ba-f511-40bf-88d5-e15d90d034b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"
Sep 08 14:14:21.352730 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.352712    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b78b6ba-f511-40bf-88d5-e15d90d034b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66\" (UID: \"9b78b6ba-f511-40bf-88d5-e15d90d034b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"
Sep 08 14:14:21.485067 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.485033    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"
Sep 08 14:14:21.773952 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:21.773924    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"]
Sep 08 14:14:21.775987 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:14:21.775958    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b78b6ba_f511_40bf_88d5_e15d90d034b6.slice/crio-6138612f72b0e6f17e9b8541dd6667cf3fe4c928c9c9909b3c6a166d01028fa4 WatchSource:0}: Error finding container 6138612f72b0e6f17e9b8541dd6667cf3fe4c928c9c9909b3c6a166d01028fa4: Status 404 returned error can't find the container with id 6138612f72b0e6f17e9b8541dd6667cf3fe4c928c9c9909b3c6a166d01028fa4
Sep 08 14:14:22.709181 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:22.709101    2564 generic.go:356] "Generic (PLEG): container finished" podID="9b78b6ba-f511-40bf-88d5-e15d90d034b6" containerID="570291283a8890dcf1a6c599852b421403435ca9ff374da7df15ba3b222a6786" exitCode=0
Sep 08 14:14:22.709181 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:22.709161    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66" event={"ID":"9b78b6ba-f511-40bf-88d5-e15d90d034b6","Type":"ContainerDied","Data":"570291283a8890dcf1a6c599852b421403435ca9ff374da7df15ba3b222a6786"}
Sep 08 14:14:22.709627 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:22.709212    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66" event={"ID":"9b78b6ba-f511-40bf-88d5-e15d90d034b6","Type":"ContainerStarted","Data":"6138612f72b0e6f17e9b8541dd6667cf3fe4c928c9c9909b3c6a166d01028fa4"}
Sep 08 14:14:23.755682 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:23.755652    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"]
Sep 08 14:14:23.756111 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:23.755887    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy" containerID="cri-o://6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df" gracePeriod=30
Sep 08 14:14:23.879883 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:23.879842    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"
Sep 08 14:14:23.980812 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:23.980730    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b78b6ba-f511-40bf-88d5-e15d90d034b6-maas-api-service-ca\") pod \"9b78b6ba-f511-40bf-88d5-e15d90d034b6\" (UID: \"9b78b6ba-f511-40bf-88d5-e15d90d034b6\") "
Sep 08 14:14:23.981127 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:23.981100    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b78b6ba-f511-40bf-88d5-e15d90d034b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b78b6ba-f511-40bf-88d5-e15d90d034b6" (UID: "9b78b6ba-f511-40bf-88d5-e15d90d034b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:14:24.082670 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:24.082631    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b78b6ba-f511-40bf-88d5-e15d90d034b6-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:24.722217 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:24.722191    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"
Sep 08 14:14:24.722433 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:24.722182    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66" event={"ID":"9b78b6ba-f511-40bf-88d5-e15d90d034b6","Type":"ContainerDied","Data":"6138612f72b0e6f17e9b8541dd6667cf3fe4c928c9c9909b3c6a166d01028fa4"}
Sep 08 14:14:24.722433 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:24.722331    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6138612f72b0e6f17e9b8541dd6667cf3fe4c928c9c9909b3c6a166d01028fa4"
Sep 08 14:14:26.332292 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:26.332261    2564 patch_prober.go:28] interesting pod/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 14:14:26.332749 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:26.332320    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 14:14:29.017573 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.017549    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:14:29.131046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.130961    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f1c296e-5446-4953-a145-419f99fa71b8-istio-podinfo\") pod \"9f1c296e-5446-4953-a145-419f99fa71b8\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") "
Sep 08 14:14:29.131046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131001    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-credential-socket\") pod \"9f1c296e-5446-4953-a145-419f99fa71b8\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") "
Sep 08 14:14:29.131265 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131053    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpjrs\" (UniqueName: \"kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-kube-api-access-jpjrs\") pod \"9f1c296e-5446-4953-a145-419f99fa71b8\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") "
Sep 08 14:14:29.131265 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131081    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f1c296e-5446-4953-a145-419f99fa71b8-istiod-ca-cert\") pod \"9f1c296e-5446-4953-a145-419f99fa71b8\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") "
Sep 08 14:14:29.131265 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131146    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-istio-token\") pod \"9f1c296e-5446-4953-a145-419f99fa71b8\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") "
Sep 08 14:14:29.131265 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131168    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-envoy\") pod \"9f1c296e-5446-4953-a145-419f99fa71b8\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") "
Sep 08 14:14:29.131265 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131194    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-socket\") pod \"9f1c296e-5446-4953-a145-419f99fa71b8\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") "
Sep 08 14:14:29.131265 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131241    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-certs\") pod \"9f1c296e-5446-4953-a145-419f99fa71b8\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") "
Sep 08 14:14:29.131557 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131331    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-data\") pod \"9f1c296e-5446-4953-a145-419f99fa71b8\" (UID: \"9f1c296e-5446-4953-a145-419f99fa71b8\") "
Sep 08 14:14:29.131557 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131366    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f1c296e-5446-4953-a145-419f99fa71b8" (UID: "9f1c296e-5446-4953-a145-419f99fa71b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:14:29.131557 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131518    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f1c296e-5446-4953-a145-419f99fa71b8" (UID: "9f1c296e-5446-4953-a145-419f99fa71b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:14:29.131709 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131607    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f1c296e-5446-4953-a145-419f99fa71b8" (UID: "9f1c296e-5446-4953-a145-419f99fa71b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:14:29.131709 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131616    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f1c296e-5446-4953-a145-419f99fa71b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f1c296e-5446-4953-a145-419f99fa71b8" (UID: "9f1c296e-5446-4953-a145-419f99fa71b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:14:29.131885 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.131735    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f1c296e-5446-4953-a145-419f99fa71b8" (UID: "9f1c296e-5446-4953-a145-419f99fa71b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:14:29.132075 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.132055    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-data\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:29.132130 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.132085    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-credential-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:29.132130 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.132101    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f1c296e-5446-4953-a145-419f99fa71b8-istiod-ca-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:29.132130 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.132115    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:29.132130 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.132128    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-workload-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:29.133321 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.133294    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f1c296e-5446-4953-a145-419f99fa71b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f1c296e-5446-4953-a145-419f99fa71b8" (UID: "9f1c296e-5446-4953-a145-419f99fa71b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:14:29.133684 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.133660    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-kube-api-access-jpjrs" (OuterVolumeSpecName: "kube-api-access-jpjrs") pod "9f1c296e-5446-4953-a145-419f99fa71b8" (UID: "9f1c296e-5446-4953-a145-419f99fa71b8"). InnerVolumeSpecName "kube-api-access-jpjrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:14:29.133880 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.133844    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f1c296e-5446-4953-a145-419f99fa71b8" (UID: "9f1c296e-5446-4953-a145-419f99fa71b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:14:29.133970 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.133925    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f1c296e-5446-4953-a145-419f99fa71b8" (UID: "9f1c296e-5446-4953-a145-419f99fa71b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:14:29.233010 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.232979    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f1c296e-5446-4953-a145-419f99fa71b8-istio-podinfo\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:29.233010 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.233007    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpjrs\" (UniqueName: \"kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-kube-api-access-jpjrs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:29.233010 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.233019    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f1c296e-5446-4953-a145-419f99fa71b8-istio-token\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:29.233227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.233028    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f1c296e-5446-4953-a145-419f99fa71b8-istio-envoy\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:29.749650 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.749617    2564 generic.go:356] "Generic (PLEG): container finished" podID="9f1c296e-5446-4953-a145-419f99fa71b8" containerID="6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df" exitCode=0
Sep 08 14:14:29.749899 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.749666    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"
Sep 08 14:14:29.749899 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.749698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" event={"ID":"9f1c296e-5446-4953-a145-419f99fa71b8","Type":"ContainerDied","Data":"6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df"}
Sep 08 14:14:29.749899 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.749760    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g" event={"ID":"9f1c296e-5446-4953-a145-419f99fa71b8","Type":"ContainerDied","Data":"fad6332a611a8b06e7b1122776c8f23cd7fcb43f09cda038c32f2636a9c15ed3"}
Sep 08 14:14:29.749899 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.749792    2564 scope.go:117] "RemoveContainer" containerID="6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df"
Sep 08 14:14:29.762087 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.761041    2564 scope.go:117] "RemoveContainer" containerID="1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b"
Sep 08 14:14:29.770384 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.770368    2564 scope.go:117] "RemoveContainer" containerID="6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df"
Sep 08 14:14:29.770652 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:14:29.770636    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df\": container with ID starting with 6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df not found: ID does not exist" containerID="6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df"
Sep 08 14:14:29.770727 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.770664    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df"} err="failed to get container status \"6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df\": rpc error: code = NotFound desc = could not find container \"6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df\": container with ID starting with 6ee92938b009948bea57032dc12338e1c8c1ffd2274eee318c90e2d4382321df not found: ID does not exist"
Sep 08 14:14:29.770727 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.770687    2564 scope.go:117] "RemoveContainer" containerID="1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b"
Sep 08 14:14:29.770985 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:14:29.770964    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b\": container with ID starting with 1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b not found: ID does not exist" containerID="1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b"
Sep 08 14:14:29.771044 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.770993    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b"} err="failed to get container status \"1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b\": rpc error: code = NotFound desc = could not find container \"1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b\": container with ID starting with 1fb9b61f2cb912eb7c899be2ce782809b90d52896fb3d302138ee6afe518dc6b not found: ID does not exist"
Sep 08 14:14:29.775819 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.775794    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"]
Sep 08 14:14:29.777854 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.777830    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-3b812d-openshift-default-6c896f555b-jgs8g"]
Sep 08 14:14:29.959795 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:29.959760    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" path="/var/lib/kubelet/pods/9f1c296e-5446-4953-a145-419f99fa71b8/volumes"
Sep 08 14:14:31.849379 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:31.849347    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"]
Sep 08 14:14:31.849792 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:31.849606    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7" podUID="85e5b001-4b4b-4392-a052-59f701c6a5bd" containerName="payload-pre-processing" containerID="cri-o://fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f" gracePeriod=30
Sep 08 14:14:32.061578 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.061548    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"]
Sep 08 14:14:32.069515 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.069486    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a47a1a97-dgf66"]
Sep 08 14:14:32.110074 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.110013    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:14:32.159628 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.159597    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkmwq\" (UniqueName: \"kubernetes.io/projected/85e5b001-4b4b-4392-a052-59f701c6a5bd-kube-api-access-bkmwq\") pod \"85e5b001-4b4b-4392-a052-59f701c6a5bd\" (UID: \"85e5b001-4b4b-4392-a052-59f701c6a5bd\") "
Sep 08 14:14:32.159821 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.159654    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85e5b001-4b4b-4392-a052-59f701c6a5bd-plugins-config-volume\") pod \"85e5b001-4b4b-4392-a052-59f701c6a5bd\" (UID: \"85e5b001-4b4b-4392-a052-59f701c6a5bd\") "
Sep 08 14:14:32.160014 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.159990    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85e5b001-4b4b-4392-a052-59f701c6a5bd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "85e5b001-4b4b-4392-a052-59f701c6a5bd" (UID: "85e5b001-4b4b-4392-a052-59f701c6a5bd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:14:32.161941 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.161841    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85e5b001-4b4b-4392-a052-59f701c6a5bd-kube-api-access-bkmwq" (OuterVolumeSpecName: "kube-api-access-bkmwq") pod "85e5b001-4b4b-4392-a052-59f701c6a5bd" (UID: "85e5b001-4b4b-4392-a052-59f701c6a5bd"). InnerVolumeSpecName "kube-api-access-bkmwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:14:32.260635 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.260600    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkmwq\" (UniqueName: \"kubernetes.io/projected/85e5b001-4b4b-4392-a052-59f701c6a5bd-kube-api-access-bkmwq\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:32.260635 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.260640    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85e5b001-4b4b-4392-a052-59f701c6a5bd-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:32.767858 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.767821    2564 generic.go:356] "Generic (PLEG): container finished" podID="85e5b001-4b4b-4392-a052-59f701c6a5bd" containerID="fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f" exitCode=0
Sep 08 14:14:32.768058 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.767886    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"
Sep 08 14:14:32.768058 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.767899    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7" event={"ID":"85e5b001-4b4b-4392-a052-59f701c6a5bd","Type":"ContainerDied","Data":"fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f"}
Sep 08 14:14:32.768058 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.767946    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7" event={"ID":"85e5b001-4b4b-4392-a052-59f701c6a5bd","Type":"ContainerDied","Data":"4791974c534886c6a69b559c254d0fed99c9fa587b5524f01abb59c1cc235259"}
Sep 08 14:14:32.768058 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.767968    2564 scope.go:117] "RemoveContainer" containerID="fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f"
Sep 08 14:14:32.778267 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.778248    2564 scope.go:117] "RemoveContainer" containerID="fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f"
Sep 08 14:14:32.778585 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:14:32.778555    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f\": container with ID starting with fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f not found: ID does not exist" containerID="fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f"
Sep 08 14:14:32.778649 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.778595    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f"} err="failed to get container status \"fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f\": rpc error: code = NotFound desc = could not find container \"fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f\": container with ID starting with fd45b3baf188e80fd7a93eb6b79725ac96160c5803c4e2d53243ef799465004f not found: ID does not exist"
Sep 08 14:14:32.790781 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.790747    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"]
Sep 08 14:14:32.807042 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:32.807014    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a47a1a97-6bc8b9b766-rpqn7"]
Sep 08 14:14:33.959312 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:33.959273    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85e5b001-4b4b-4392-a052-59f701c6a5bd" path="/var/lib/kubelet/pods/85e5b001-4b4b-4392-a052-59f701c6a5bd/volumes"
Sep 08 14:14:33.959709 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:33.959609    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b78b6ba-f511-40bf-88d5-e15d90d034b6" path="/var/lib/kubelet/pods/9b78b6ba-f511-40bf-88d5-e15d90d034b6/volumes"
Sep 08 14:14:39.312046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:39.312015    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"]
Sep 08 14:14:39.312546 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:39.312222    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy" containerID="cri-o://66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87" gracePeriod=30
Sep 08 14:14:43.012799 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:43.012759    2564 patch_prober.go:28] interesting pod/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 14:14:43.013192 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:43.012817    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 14:14:44.558175 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.558147    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:14:44.684672 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.684588    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-certs\") pod \"a35854c5-2d99-4a16-911d-aa17eda1e21d\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") "
Sep 08 14:14:44.684672 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.684641    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-token\") pod \"a35854c5-2d99-4a16-911d-aa17eda1e21d\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") "
Sep 08 14:14:44.684672 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.684671    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-socket\") pod \"a35854c5-2d99-4a16-911d-aa17eda1e21d\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") "
Sep 08 14:14:44.684993 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.684691    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-podinfo\") pod \"a35854c5-2d99-4a16-911d-aa17eda1e21d\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") "
Sep 08 14:14:44.684993 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.684723    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a35854c5-2d99-4a16-911d-aa17eda1e21d-istiod-ca-cert\") pod \"a35854c5-2d99-4a16-911d-aa17eda1e21d\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") "
Sep 08 14:14:44.684993 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.684771    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4g7n6\" (UniqueName: \"kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-kube-api-access-4g7n6\") pod \"a35854c5-2d99-4a16-911d-aa17eda1e21d\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") "
Sep 08 14:14:44.684993 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.684807    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-data\") pod \"a35854c5-2d99-4a16-911d-aa17eda1e21d\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") "
Sep 08 14:14:44.684993 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.684846    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-envoy\") pod \"a35854c5-2d99-4a16-911d-aa17eda1e21d\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") "
Sep 08 14:14:44.684993 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.684914    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-credential-socket\") pod \"a35854c5-2d99-4a16-911d-aa17eda1e21d\" (UID: \"a35854c5-2d99-4a16-911d-aa17eda1e21d\") "
Sep 08 14:14:44.684993 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.684928    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a35854c5-2d99-4a16-911d-aa17eda1e21d" (UID: "a35854c5-2d99-4a16-911d-aa17eda1e21d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:14:44.685360 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.685056    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a35854c5-2d99-4a16-911d-aa17eda1e21d" (UID: "a35854c5-2d99-4a16-911d-aa17eda1e21d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:14:44.685360 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.685274    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:44.685360 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.685275    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-data" (OuterVolumeSpecName: "istio-data") pod "a35854c5-2d99-4a16-911d-aa17eda1e21d" (UID: "a35854c5-2d99-4a16-911d-aa17eda1e21d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:14:44.685360 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.685294    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-workload-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:44.685360 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.685299    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a35854c5-2d99-4a16-911d-aa17eda1e21d" (UID: "a35854c5-2d99-4a16-911d-aa17eda1e21d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:14:44.685798 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.685403    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a35854c5-2d99-4a16-911d-aa17eda1e21d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a35854c5-2d99-4a16-911d-aa17eda1e21d" (UID: "a35854c5-2d99-4a16-911d-aa17eda1e21d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:14:44.687360 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.687334    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-kube-api-access-4g7n6" (OuterVolumeSpecName: "kube-api-access-4g7n6") pod "a35854c5-2d99-4a16-911d-aa17eda1e21d" (UID: "a35854c5-2d99-4a16-911d-aa17eda1e21d"). InnerVolumeSpecName "kube-api-access-4g7n6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:14:44.687635 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.687607    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-token" (OuterVolumeSpecName: "istio-token") pod "a35854c5-2d99-4a16-911d-aa17eda1e21d" (UID: "a35854c5-2d99-4a16-911d-aa17eda1e21d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:14:44.687762 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.687651    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a35854c5-2d99-4a16-911d-aa17eda1e21d" (UID: "a35854c5-2d99-4a16-911d-aa17eda1e21d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:14:44.687762 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.687666    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a35854c5-2d99-4a16-911d-aa17eda1e21d" (UID: "a35854c5-2d99-4a16-911d-aa17eda1e21d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:14:44.785805 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.785771    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4g7n6\" (UniqueName: \"kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-kube-api-access-4g7n6\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:44.785805 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.785803    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-data\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:44.785805 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.785812    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-envoy\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:44.786070 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.785820    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a35854c5-2d99-4a16-911d-aa17eda1e21d-credential-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:44.786070 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.785828    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-token\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:44.786070 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.785836    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a35854c5-2d99-4a16-911d-aa17eda1e21d-istio-podinfo\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:44.786070 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.785843    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a35854c5-2d99-4a16-911d-aa17eda1e21d-istiod-ca-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:14:44.845764 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.845730    2564 generic.go:356] "Generic (PLEG): container finished" podID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerID="66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87" exitCode=0
Sep 08 14:14:44.845958 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.845780    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" event={"ID":"a35854c5-2d99-4a16-911d-aa17eda1e21d","Type":"ContainerDied","Data":"66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87"}
Sep 08 14:14:44.845958 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.845784    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"
Sep 08 14:14:44.845958 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.845820    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb" event={"ID":"a35854c5-2d99-4a16-911d-aa17eda1e21d","Type":"ContainerDied","Data":"fb14b62e6f3ac92dce0452a19b94ead5411f9e46ba9b1aff0bbe0944399d693a"}
Sep 08 14:14:44.845958 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.845844    2564 scope.go:117] "RemoveContainer" containerID="66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87"
Sep 08 14:14:44.856218 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.856201    2564 scope.go:117] "RemoveContainer" containerID="0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b"
Sep 08 14:14:44.864723 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.864698    2564 scope.go:117] "RemoveContainer" containerID="66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87"
Sep 08 14:14:44.865030 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:14:44.865010    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87\": container with ID starting with 66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87 not found: ID does not exist" containerID="66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87"
Sep 08 14:14:44.865124 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.865042    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87"} err="failed to get container status \"66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87\": rpc error: code = NotFound desc = could not find container \"66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87\": container with ID starting with 66749223c565556de652ea8a4861e562fe73730b731baea7b9aa4c1cb09e2c87 not found: ID does not exist"
Sep 08 14:14:44.865124 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.865067    2564 scope.go:117] "RemoveContainer" containerID="0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b"
Sep 08 14:14:44.865326 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:14:44.865308    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b\": container with ID starting with 0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b not found: ID does not exist" containerID="0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b"
Sep 08 14:14:44.865369 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.865333    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b"} err="failed to get container status \"0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b\": rpc error: code = NotFound desc = could not find container \"0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b\": container with ID starting with 0e255a84c01f1df424d023b566c3f0357b51a9b706bd13d2cd6f9bd9109b7d6b not found: ID does not exist"
Sep 08 14:14:44.871818 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.871795    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"]
Sep 08 14:14:44.876436 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:44.876415    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a47a1a97-openshift-default-5d9d4b5576-47dtb"]
Sep 08 14:14:45.967049 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:14:45.967012    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" path="/var/lib/kubelet/pods/a35854c5-2d99-4a16-911d-aa17eda1e21d/volumes"
Sep 08 14:15:00.215355 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.215260    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr"]
Sep 08 14:15:00.215844 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.215827    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy"
Sep 08 14:15:00.215910 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.215848    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy"
Sep 08 14:15:00.215910 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.215898    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy"
Sep 08 14:15:00.215910 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.215908    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy"
Sep 08 14:15:00.216011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.215921    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy"
Sep 08 14:15:00.216011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.215929    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy"
Sep 08 14:15:00.216011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.215939    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b78b6ba-f511-40bf-88d5-e15d90d034b6" containerName="revoke-keys"
Sep 08 14:15:00.216011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.215947    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b78b6ba-f511-40bf-88d5-e15d90d034b6" containerName="revoke-keys"
Sep 08 14:15:00.216011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.215970    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy"
Sep 08 14:15:00.216011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.215978    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy"
Sep 08 14:15:00.216011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.216000    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85e5b001-4b4b-4392-a052-59f701c6a5bd" containerName="payload-pre-processing"
Sep 08 14:15:00.216011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.216009    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="85e5b001-4b4b-4392-a052-59f701c6a5bd" containerName="payload-pre-processing"
Sep 08 14:15:00.216250 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.216110    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy"
Sep 08 14:15:00.216250 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.216120    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="85e5b001-4b4b-4392-a052-59f701c6a5bd" containerName="payload-pre-processing"
Sep 08 14:15:00.216250 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.216128    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a35854c5-2d99-4a16-911d-aa17eda1e21d" containerName="istio-proxy"
Sep 08 14:15:00.216250 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.216140    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy"
Sep 08 14:15:00.216250 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.216151    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b78b6ba-f511-40bf-88d5-e15d90d034b6" containerName="revoke-keys"
Sep 08 14:15:00.219562 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.219546    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq"]
Sep 08 14:15:00.219677 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.219663    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr"
Sep 08 14:15:00.220106 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.220090    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f1c296e-5446-4953-a145-419f99fa71b8" containerName="istio-proxy"
Sep 08 14:15:00.222694 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.222676    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq"
Sep 08 14:15:00.224505 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.224486    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:15:00.230562 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.230539    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr"
Sep 08 14:15:00.236727 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.236709    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq"
Sep 08 14:15:00.245795 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.245768    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq"]
Sep 08 14:15:00.250759 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.250727    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr"]
Sep 08 14:15:00.539105 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.538952    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr"]
Sep 08 14:15:00.541753 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.541722    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq"]
Sep 08 14:15:00.542697 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:15:00.542669    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869e083c_e354_4050_92f7_9129ac7ca466.slice/crio-24ad5112df29bf65b93bbd4b284d063d2cea43c7614d34e42907fe95f0a04fae WatchSource:0}: Error finding container 24ad5112df29bf65b93bbd4b284d063d2cea43c7614d34e42907fe95f0a04fae: Status 404 returned error can't find the container with id 24ad5112df29bf65b93bbd4b284d063d2cea43c7614d34e42907fe95f0a04fae
Sep 08 14:15:00.543367 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:15:00.543343    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd4357e_9df0_4e85_97b2_dc586f78c170.slice/crio-f9ddffc4f2bdbc1b2fae8b21ddbd49f1bf2fc88adaf6c9cbddbbb25e091b0d70 WatchSource:0}: Error finding container f9ddffc4f2bdbc1b2fae8b21ddbd49f1bf2fc88adaf6c9cbddbbb25e091b0d70: Status 404 returned error can't find the container with id f9ddffc4f2bdbc1b2fae8b21ddbd49f1bf2fc88adaf6c9cbddbbb25e091b0d70
Sep 08 14:15:00.938996 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.938968    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr" event={"ID":"869e083c-e354-4050-92f7-9129ac7ca466","Type":"ContainerStarted","Data":"d63558a431f6a9e1737801cd0d06210121d57fb084e5626eea5c4f333574ba71"}
Sep 08 14:15:00.939133 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.939032    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr" event={"ID":"869e083c-e354-4050-92f7-9129ac7ca466","Type":"ContainerStarted","Data":"24ad5112df29bf65b93bbd4b284d063d2cea43c7614d34e42907fe95f0a04fae"}
Sep 08 14:15:00.940544 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.940519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq" event={"ID":"add4357e-9df0-4e85-97b2-dc586f78c170","Type":"ContainerStarted","Data":"f9ddffc4f2bdbc1b2fae8b21ddbd49f1bf2fc88adaf6c9cbddbbb25e091b0d70"}
Sep 08 14:15:00.997430 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.997375    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr" podStartSLOduration=0.997358936 podStartE2EDuration="997.358936ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:15:00.972046506 +0000 UTC m=+991.467017191" watchObservedRunningTime="2026-09-08 14:15:00.997358936 +0000 UTC m=+991.492329619"
Sep 08 14:15:00.997766 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:00.997736    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq" podStartSLOduration=0.997727978 podStartE2EDuration="997.727978ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:15:00.995090027 +0000 UTC m=+991.490060710" watchObservedRunningTime="2026-09-08 14:15:00.997727978 +0000 UTC m=+991.492698661"
Sep 08 14:15:01.805914 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.805791    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"]
Sep 08 14:15:01.809467 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.809445    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:15:01.813539 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.813517    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-2bcb20\""
Sep 08 14:15:01.813654 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.813581    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-2bcb20-dockercfg-df49b\""
Sep 08 14:15:01.832643 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.832615    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"]
Sep 08 14:15:01.834775 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.834752    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"]
Sep 08 14:15:01.838621 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.838600    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:15:01.850467 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.850066    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d275\" (UniqueName: \"kubernetes.io/projected/bf2fb49b-e774-41c8-8ea4-14daa18591e8-kube-api-access-6d275\") pod \"payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2\" (UID: \"bf2fb49b-e774-41c8-8ea4-14daa18591e8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:15:01.850467 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.850116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8xkr\" (UniqueName: \"kubernetes.io/projected/576a03f2-3518-4a1b-bdfa-858583427e84-kube-api-access-j8xkr\") pod \"payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl\" (UID: \"576a03f2-3518-4a1b-bdfa-858583427e84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:15:01.850467 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.850174    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf2fb49b-e774-41c8-8ea4-14daa18591e8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2\" (UID: \"bf2fb49b-e774-41c8-8ea4-14daa18591e8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:15:01.850467 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.850209    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/576a03f2-3518-4a1b-bdfa-858583427e84-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl\" (UID: \"576a03f2-3518-4a1b-bdfa-858583427e84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:15:01.853037 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.853017    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"]
Sep 08 14:15:01.949038 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.949003    2564 generic.go:356] "Generic (PLEG): container finished" podID="869e083c-e354-4050-92f7-9129ac7ca466" containerID="d63558a431f6a9e1737801cd0d06210121d57fb084e5626eea5c4f333574ba71" exitCode=0
Sep 08 14:15:01.949230 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.949089    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr" event={"ID":"869e083c-e354-4050-92f7-9129ac7ca466","Type":"ContainerDied","Data":"d63558a431f6a9e1737801cd0d06210121d57fb084e5626eea5c4f333574ba71"}
Sep 08 14:15:01.950757 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.950732    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d275\" (UniqueName: \"kubernetes.io/projected/bf2fb49b-e774-41c8-8ea4-14daa18591e8-kube-api-access-6d275\") pod \"payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2\" (UID: \"bf2fb49b-e774-41c8-8ea4-14daa18591e8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:15:01.950923 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.950775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8xkr\" (UniqueName: \"kubernetes.io/projected/576a03f2-3518-4a1b-bdfa-858583427e84-kube-api-access-j8xkr\") pod \"payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl\" (UID: \"576a03f2-3518-4a1b-bdfa-858583427e84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:15:01.950923 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.950836    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf2fb49b-e774-41c8-8ea4-14daa18591e8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2\" (UID: \"bf2fb49b-e774-41c8-8ea4-14daa18591e8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:15:01.950923 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.950903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/576a03f2-3518-4a1b-bdfa-858583427e84-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl\" (UID: \"576a03f2-3518-4a1b-bdfa-858583427e84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:15:01.950923 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.950912    2564 generic.go:356] "Generic (PLEG): container finished" podID="add4357e-9df0-4e85-97b2-dc586f78c170" containerID="8aa82ec3492f47813eab84aa962c5d9e3b956e16eaf1f491067db011f1cc902d" exitCode=0
Sep 08 14:15:01.951147 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.950978    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq" event={"ID":"add4357e-9df0-4e85-97b2-dc586f78c170","Type":"ContainerDied","Data":"8aa82ec3492f47813eab84aa962c5d9e3b956e16eaf1f491067db011f1cc902d"}
Sep 08 14:15:01.951555 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.951537    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf2fb49b-e774-41c8-8ea4-14daa18591e8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2\" (UID: \"bf2fb49b-e774-41c8-8ea4-14daa18591e8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:15:01.951627 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.951604    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/576a03f2-3518-4a1b-bdfa-858583427e84-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl\" (UID: \"576a03f2-3518-4a1b-bdfa-858583427e84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:15:01.970424 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.970394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d275\" (UniqueName: \"kubernetes.io/projected/bf2fb49b-e774-41c8-8ea4-14daa18591e8-kube-api-access-6d275\") pod \"payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2\" (UID: \"bf2fb49b-e774-41c8-8ea4-14daa18591e8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:15:01.970562 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:01.970478    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8xkr\" (UniqueName: \"kubernetes.io/projected/576a03f2-3518-4a1b-bdfa-858583427e84-kube-api-access-j8xkr\") pod \"payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl\" (UID: \"576a03f2-3518-4a1b-bdfa-858583427e84\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:15:02.120063 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:02.119961    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:15:02.148398 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:02.148367    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:15:02.405117 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:02.405087    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"]
Sep 08 14:15:02.405841 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:15:02.405816    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod576a03f2_3518_4a1b_bdfa_858583427e84.slice/crio-2bd6c2866fcc81dc7674ddec9e5400a392e097f113dbf07151b1050c34ee5854 WatchSource:0}: Error finding container 2bd6c2866fcc81dc7674ddec9e5400a392e097f113dbf07151b1050c34ee5854: Status 404 returned error can't find the container with id 2bd6c2866fcc81dc7674ddec9e5400a392e097f113dbf07151b1050c34ee5854
Sep 08 14:15:02.438156 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:02.438129    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"]
Sep 08 14:15:02.441955 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:15:02.441919    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf2fb49b_e774_41c8_8ea4_14daa18591e8.slice/crio-95ec62914e8b662c6bad4b75b94a181314f288f9091bd4eb7f640a74ee390b36 WatchSource:0}: Error finding container 95ec62914e8b662c6bad4b75b94a181314f288f9091bd4eb7f640a74ee390b36: Status 404 returned error can't find the container with id 95ec62914e8b662c6bad4b75b94a181314f288f9091bd4eb7f640a74ee390b36
Sep 08 14:15:02.959053 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:02.959018    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl" event={"ID":"576a03f2-3518-4a1b-bdfa-858583427e84","Type":"ContainerStarted","Data":"b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8"}
Sep 08 14:15:02.959548 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:02.959080    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl" event={"ID":"576a03f2-3518-4a1b-bdfa-858583427e84","Type":"ContainerStarted","Data":"2bd6c2866fcc81dc7674ddec9e5400a392e097f113dbf07151b1050c34ee5854"}
Sep 08 14:15:02.959548 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:02.959296    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:15:02.960956 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:02.960898    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2" event={"ID":"bf2fb49b-e774-41c8-8ea4-14daa18591e8","Type":"ContainerStarted","Data":"e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9"}
Sep 08 14:15:02.960956 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:02.960951    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2" event={"ID":"bf2fb49b-e774-41c8-8ea4-14daa18591e8","Type":"ContainerStarted","Data":"95ec62914e8b662c6bad4b75b94a181314f288f9091bd4eb7f640a74ee390b36"}
Sep 08 14:15:02.961179 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:02.961064    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:15:02.987003 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:02.986841    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl" podStartSLOduration=1.986816419 podStartE2EDuration="1.986816419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:15:02.986099321 +0000 UTC m=+993.481070007" watchObservedRunningTime="2026-09-08 14:15:02.986816419 +0000 UTC m=+993.481787126"
Sep 08 14:15:03.013270 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:03.013211    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2" podStartSLOduration=2.013193014 podStartE2EDuration="2.013193014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:15:03.011516831 +0000 UTC m=+993.506487516" watchObservedRunningTime="2026-09-08 14:15:03.013193014 +0000 UTC m=+993.508163697"
Sep 08 14:15:03.110172 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:03.110145    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr"
Sep 08 14:15:03.125997 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:03.125160    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq"
Sep 08 14:15:03.968081 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:03.968049    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr"
Sep 08 14:15:03.968562 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:03.968083    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr" event={"ID":"869e083c-e354-4050-92f7-9129ac7ca466","Type":"ContainerDied","Data":"24ad5112df29bf65b93bbd4b284d063d2cea43c7614d34e42907fe95f0a04fae"}
Sep 08 14:15:03.968562 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:03.968131    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24ad5112df29bf65b93bbd4b284d063d2cea43c7614d34e42907fe95f0a04fae"
Sep 08 14:15:03.970722 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:03.970590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq" event={"ID":"add4357e-9df0-4e85-97b2-dc586f78c170","Type":"ContainerDied","Data":"f9ddffc4f2bdbc1b2fae8b21ddbd49f1bf2fc88adaf6c9cbddbbb25e091b0d70"}
Sep 08 14:15:03.970722 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:03.970649    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9ddffc4f2bdbc1b2fae8b21ddbd49f1bf2fc88adaf6c9cbddbbb25e091b0d70"
Sep 08 14:15:03.970722 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:03.970701    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq"
Sep 08 14:15:13.972675 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:13.972643    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:15:13.973107 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:13.972733    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:15:20.130350 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.130303    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"]
Sep 08 14:15:20.130960 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.130704    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="869e083c-e354-4050-92f7-9129ac7ca466" containerName="cleanup"
Sep 08 14:15:20.130960 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.130721    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="869e083c-e354-4050-92f7-9129ac7ca466" containerName="cleanup"
Sep 08 14:15:20.130960 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.130735    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="add4357e-9df0-4e85-97b2-dc586f78c170" containerName="cleanup"
Sep 08 14:15:20.130960 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.130741    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="add4357e-9df0-4e85-97b2-dc586f78c170" containerName="cleanup"
Sep 08 14:15:20.130960 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.130814    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="add4357e-9df0-4e85-97b2-dc586f78c170" containerName="cleanup"
Sep 08 14:15:20.130960 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.130825    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="869e083c-e354-4050-92f7-9129ac7ca466" containerName="cleanup"
Sep 08 14:15:20.134152 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.134132    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"
Sep 08 14:15:20.136667 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.136647    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:15:20.137176 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.137154    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:15:20.142392 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.142289    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"]
Sep 08 14:15:20.232154 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.232124    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1468079b-76bc-4c7b-a5ae-3061f95488d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc\" (UID: \"1468079b-76bc-4c7b-a5ae-3061f95488d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"
Sep 08 14:15:20.332659 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.332629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1468079b-76bc-4c7b-a5ae-3061f95488d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc\" (UID: \"1468079b-76bc-4c7b-a5ae-3061f95488d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"
Sep 08 14:15:20.333391 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.333360    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1468079b-76bc-4c7b-a5ae-3061f95488d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc\" (UID: \"1468079b-76bc-4c7b-a5ae-3061f95488d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"
Sep 08 14:15:20.446282 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.446185    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"
Sep 08 14:15:20.722815 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:20.722786    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"]
Sep 08 14:15:20.724325 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:15:20.724290    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1468079b_76bc_4c7b_a5ae_3061f95488d1.slice/crio-5844ca44fa0f898c087a0cffeee661cb5104a380665e9b73ce61500e4d7d2030 WatchSource:0}: Error finding container 5844ca44fa0f898c087a0cffeee661cb5104a380665e9b73ce61500e4d7d2030: Status 404 returned error can't find the container with id 5844ca44fa0f898c087a0cffeee661cb5104a380665e9b73ce61500e4d7d2030
Sep 08 14:15:21.067957 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:21.067845    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc" event={"ID":"1468079b-76bc-4c7b-a5ae-3061f95488d1","Type":"ContainerStarted","Data":"51369b6276bfed39b59193ca7e42e9f93f53a7c85446136f9c137aa7bf0533a8"}
Sep 08 14:15:21.067957 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:21.067923    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc" event={"ID":"1468079b-76bc-4c7b-a5ae-3061f95488d1","Type":"ContainerStarted","Data":"5844ca44fa0f898c087a0cffeee661cb5104a380665e9b73ce61500e4d7d2030"}
Sep 08 14:15:21.089308 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:21.089255    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc" podStartSLOduration=1.089236728 podStartE2EDuration="1.089236728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:15:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:15:21.087694092 +0000 UTC m=+1011.582664776" watchObservedRunningTime="2026-09-08 14:15:21.089236728 +0000 UTC m=+1011.584207415"
Sep 08 14:15:22.076936 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:22.076888    2564 generic.go:356] "Generic (PLEG): container finished" podID="1468079b-76bc-4c7b-a5ae-3061f95488d1" containerID="51369b6276bfed39b59193ca7e42e9f93f53a7c85446136f9c137aa7bf0533a8" exitCode=0
Sep 08 14:15:22.077308 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:22.076958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc" event={"ID":"1468079b-76bc-4c7b-a5ae-3061f95488d1","Type":"ContainerDied","Data":"51369b6276bfed39b59193ca7e42e9f93f53a7c85446136f9c137aa7bf0533a8"}
Sep 08 14:15:23.216594 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:23.216569    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"
Sep 08 14:15:23.363425 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:23.363325    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1468079b-76bc-4c7b-a5ae-3061f95488d1-maas-api-service-ca\") pod \"1468079b-76bc-4c7b-a5ae-3061f95488d1\" (UID: \"1468079b-76bc-4c7b-a5ae-3061f95488d1\") "
Sep 08 14:15:23.363693 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:23.363669    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1468079b-76bc-4c7b-a5ae-3061f95488d1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1468079b-76bc-4c7b-a5ae-3061f95488d1" (UID: "1468079b-76bc-4c7b-a5ae-3061f95488d1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:15:23.465010 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:23.464975    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1468079b-76bc-4c7b-a5ae-3061f95488d1-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:24.089485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:24.089442    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc" event={"ID":"1468079b-76bc-4c7b-a5ae-3061f95488d1","Type":"ContainerDied","Data":"5844ca44fa0f898c087a0cffeee661cb5104a380665e9b73ce61500e4d7d2030"}
Sep 08 14:15:24.089485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:24.089468    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"
Sep 08 14:15:24.089712 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:24.089500    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5844ca44fa0f898c087a0cffeee661cb5104a380665e9b73ce61500e4d7d2030"
Sep 08 14:15:25.073345 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.073315    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"]
Sep 08 14:15:25.073761 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.073747    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1468079b-76bc-4c7b-a5ae-3061f95488d1" containerName="revoke-keys"
Sep 08 14:15:25.073820 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.073762    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1468079b-76bc-4c7b-a5ae-3061f95488d1" containerName="revoke-keys"
Sep 08 14:15:25.073856 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.073824    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1468079b-76bc-4c7b-a5ae-3061f95488d1" containerName="revoke-keys"
Sep 08 14:15:25.077192 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.077168    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.081350 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.081333    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-e2a04d-openshift-default-dockercfg-gfbnw\""
Sep 08 14:15:25.098406 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.098378    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"]
Sep 08 14:15:25.183399 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.183366    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-token\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.183574 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.183419    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-credential-socket\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.183574 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.183465    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-certs\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.183574 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.183483    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-envoy\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.183574 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.183501    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-data\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.183574 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.183553    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6zzw\" (UniqueName: \"kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-kube-api-access-h6zzw\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.183752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.183625    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-podinfo\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.183752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.183646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-socket\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.183752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.183718    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee5f985-7473-447b-a1f3-d886935ae0bc-istiod-ca-cert\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.284656 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.284617    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-token\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.284859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.284678    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-credential-socket\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.284859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.284721    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-certs\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.284859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.284749    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-envoy\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.284859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.284773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-data\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.284859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.284803    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6zzw\" (UniqueName: \"kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-kube-api-access-h6zzw\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.284859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.284859    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-podinfo\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.285227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.284915    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-socket\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.285227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.284981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee5f985-7473-447b-a1f3-d886935ae0bc-istiod-ca-cert\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.285227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.285114    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-credential-socket\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.285227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.285173    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-certs\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.285448 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.285425    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-socket\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.285656 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.285633    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-data\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.285896 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.285857    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee5f985-7473-447b-a1f3-d886935ae0bc-istiod-ca-cert\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.287285 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.287268    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-envoy\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.287613 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.287598    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-podinfo\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.297736 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.297711    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-token\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.298182 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.298161    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6zzw\" (UniqueName: \"kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-kube-api-access-h6zzw\") pod \"e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") " pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.393214 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.393115    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:25.677137 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.677112    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"]
Sep 08 14:15:25.678746 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:15:25.678720    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee5f985_7473_447b_a1f3_d886935ae0bc.slice/crio-40e167a8817ef71dab54847797a0a514ed4cd56fd3f4647ad03e0321d38f5b73 WatchSource:0}: Error finding container 40e167a8817ef71dab54847797a0a514ed4cd56fd3f4647ad03e0321d38f5b73: Status 404 returned error can't find the container with id 40e167a8817ef71dab54847797a0a514ed4cd56fd3f4647ad03e0321d38f5b73
Sep 08 14:15:25.681212 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.681177    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:15:25.681276 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.681250    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:15:25.681316 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:25.681279    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:15:26.104976 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:26.104938    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" event={"ID":"0ee5f985-7473-447b-a1f3-d886935ae0bc","Type":"ContainerStarted","Data":"8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c"}
Sep 08 14:15:26.105383 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:26.104994    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" event={"ID":"0ee5f985-7473-447b-a1f3-d886935ae0bc","Type":"ContainerStarted","Data":"40e167a8817ef71dab54847797a0a514ed4cd56fd3f4647ad03e0321d38f5b73"}
Sep 08 14:15:26.133245 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:26.133198    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" podStartSLOduration=1.133181202 podStartE2EDuration="1.133181202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:15:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:15:26.129753277 +0000 UTC m=+1016.624723962" watchObservedRunningTime="2026-09-08 14:15:26.133181202 +0000 UTC m=+1016.628151886"
Sep 08 14:15:26.394289 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:26.394194    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:26.399429 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:26.399406    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:27.111459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:27.111429    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:27.112615 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:27.112595    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:30.313950 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:30.313851    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq"]
Sep 08 14:15:30.318614 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:30.318590    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cb068d-29814615-bv4fq"]
Sep 08 14:15:30.763741 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:30.763703    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"]
Sep 08 14:15:30.769356 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:30.769328    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-cb068d-q9gzc"]
Sep 08 14:15:31.958630 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:31.958594    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1468079b-76bc-4c7b-a5ae-3061f95488d1" path="/var/lib/kubelet/pods/1468079b-76bc-4c7b-a5ae-3061f95488d1/volumes"
Sep 08 14:15:31.959140 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:31.959024    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="add4357e-9df0-4e85-97b2-dc586f78c170" path="/var/lib/kubelet/pods/add4357e-9df0-4e85-97b2-dc586f78c170/volumes"
Sep 08 14:15:37.982052 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:37.982017    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"]
Sep 08 14:15:37.982528 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:37.982292    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy" containerID="cri-o://e91b2138d980ff667a7252148d1734d5237864ded03c880e1a3168a595c67c7a" gracePeriod=30
Sep 08 14:15:38.329514 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:38.329434    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"]
Sep 08 14:15:38.334368 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:38.334349    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"
Sep 08 14:15:38.337356 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:38.337334    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:15:38.337475 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:38.337385    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:15:38.342887 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:38.342825    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"]
Sep 08 14:15:38.517179 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:38.517143    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc\" (UID: \"aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"
Sep 08 14:15:38.618306 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:38.618201    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc\" (UID: \"aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"
Sep 08 14:15:38.618930 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:38.618904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc\" (UID: \"aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"
Sep 08 14:15:38.647124 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:38.647091    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"
Sep 08 14:15:38.924524 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:38.924491    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"]
Sep 08 14:15:38.925917 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:15:38.925856    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaabdfb7a_d6f5_4c94_b6c9_6d3c586338b8.slice/crio-45a283e90b054335b09609f1ee8796a8c8d28b9949a3b1656be6d926f14f287d WatchSource:0}: Error finding container 45a283e90b054335b09609f1ee8796a8c8d28b9949a3b1656be6d926f14f287d: Status 404 returned error can't find the container with id 45a283e90b054335b09609f1ee8796a8c8d28b9949a3b1656be6d926f14f287d
Sep 08 14:15:39.194815 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:39.194722    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc" event={"ID":"aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8","Type":"ContainerStarted","Data":"bbbff0c0f6559bbb16ce431f260ae073c40f5cf68fb34db6b845e3b36ad02ea4"}
Sep 08 14:15:39.195181 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:39.194821    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc" event={"ID":"aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8","Type":"ContainerStarted","Data":"45a283e90b054335b09609f1ee8796a8c8d28b9949a3b1656be6d926f14f287d"}
Sep 08 14:15:39.212158 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:39.212099    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc" podStartSLOduration=1.212078124 podStartE2EDuration="1.212078124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:15:39.210243692 +0000 UTC m=+1029.705214370" watchObservedRunningTime="2026-09-08 14:15:39.212078124 +0000 UTC m=+1029.707048809"
Sep 08 14:15:39.685555 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:39.685517    2564 patch_prober.go:28] interesting pod/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 14:15:39.685699 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:39.685588    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 14:15:40.203212 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:40.203177    2564 generic.go:356] "Generic (PLEG): container finished" podID="aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8" containerID="bbbff0c0f6559bbb16ce431f260ae073c40f5cf68fb34db6b845e3b36ad02ea4" exitCode=0
Sep 08 14:15:40.203647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:40.203320    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc" event={"ID":"aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8","Type":"ContainerDied","Data":"bbbff0c0f6559bbb16ce431f260ae073c40f5cf68fb34db6b845e3b36ad02ea4"}
Sep 08 14:15:41.344470 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:41.344446    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"
Sep 08 14:15:41.442502 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:41.442467    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8-maas-api-service-ca\") pod \"aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8\" (UID: \"aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8\") "
Sep 08 14:15:41.442898 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:41.442846    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8" (UID: "aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:15:41.543320 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:41.543286    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:42.216670 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:42.216590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc" event={"ID":"aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8","Type":"ContainerDied","Data":"45a283e90b054335b09609f1ee8796a8c8d28b9949a3b1656be6d926f14f287d"}
Sep 08 14:15:42.216670 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:42.216623    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"
Sep 08 14:15:42.216670 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:42.216641    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45a283e90b054335b09609f1ee8796a8c8d28b9949a3b1656be6d926f14f287d"
Sep 08 14:15:43.243513 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.243474    2564 generic.go:356] "Generic (PLEG): container finished" podID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerID="e91b2138d980ff667a7252148d1734d5237864ded03c880e1a3168a595c67c7a" exitCode=0
Sep 08 14:15:43.244193 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.244166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" event={"ID":"ca21c26b-4510-498d-819f-2bc6fa4a2dae","Type":"ContainerDied","Data":"e91b2138d980ff667a7252148d1734d5237864ded03c880e1a3168a595c67c7a"}
Sep 08 14:15:43.244377 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.244361    2564 scope.go:117] "RemoveContainer" containerID="2ece0fdd579d060cca9093b0c399fc36ab997de41732a1c215d97dc05eaf2ab0"
Sep 08 14:15:43.283601 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.283579    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:15:43.360233 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360195    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-km9xq\" (UniqueName: \"kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-kube-api-access-km9xq\") pod \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") "
Sep 08 14:15:43.360395 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360267    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-credential-socket\") pod \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") "
Sep 08 14:15:43.360395 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360290    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-data\") pod \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") "
Sep 08 14:15:43.360395 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360338    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-socket\") pod \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") "
Sep 08 14:15:43.360395 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360383    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istiod-ca-cert\") pod \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") "
Sep 08 14:15:43.360623 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360401    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-certs\") pod \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") "
Sep 08 14:15:43.360623 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360415    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-envoy\") pod \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") "
Sep 08 14:15:43.360623 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360439    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-podinfo\") pod \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") "
Sep 08 14:15:43.360623 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360472    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-token\") pod \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\" (UID: \"ca21c26b-4510-498d-819f-2bc6fa4a2dae\") "
Sep 08 14:15:43.360833 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360670    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ca21c26b-4510-498d-819f-2bc6fa4a2dae" (UID: "ca21c26b-4510-498d-819f-2bc6fa4a2dae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:15:43.360833 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360673    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-data" (OuterVolumeSpecName: "istio-data") pod "ca21c26b-4510-498d-819f-2bc6fa4a2dae" (UID: "ca21c26b-4510-498d-819f-2bc6fa4a2dae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:15:43.360833 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360688    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ca21c26b-4510-498d-819f-2bc6fa4a2dae" (UID: "ca21c26b-4510-498d-819f-2bc6fa4a2dae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:15:43.360833 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360703    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ca21c26b-4510-498d-819f-2bc6fa4a2dae" (UID: "ca21c26b-4510-498d-819f-2bc6fa4a2dae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:15:43.360833 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360753    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ca21c26b-4510-498d-819f-2bc6fa4a2dae" (UID: "ca21c26b-4510-498d-819f-2bc6fa4a2dae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:15:43.361074 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360841    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-credential-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:43.361074 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360880    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-data\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:43.361074 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360904    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:43.361074 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360919    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istiod-ca-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:43.361074 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.360933    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-workload-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:43.363382 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.363351    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-token" (OuterVolumeSpecName: "istio-token") pod "ca21c26b-4510-498d-819f-2bc6fa4a2dae" (UID: "ca21c26b-4510-498d-819f-2bc6fa4a2dae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:15:43.363542 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.363514    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ca21c26b-4510-498d-819f-2bc6fa4a2dae" (UID: "ca21c26b-4510-498d-819f-2bc6fa4a2dae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:15:43.363602 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.363528    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-kube-api-access-km9xq" (OuterVolumeSpecName: "kube-api-access-km9xq") pod "ca21c26b-4510-498d-819f-2bc6fa4a2dae" (UID: "ca21c26b-4510-498d-819f-2bc6fa4a2dae"). InnerVolumeSpecName "kube-api-access-km9xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:15:43.363602 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.363571    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ca21c26b-4510-498d-819f-2bc6fa4a2dae" (UID: "ca21c26b-4510-498d-819f-2bc6fa4a2dae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:15:43.461390 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.461354    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-envoy\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:43.461390 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.461385    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-podinfo\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:43.461390 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.461395    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-istio-token\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:43.461635 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:43.461403    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-km9xq\" (UniqueName: \"kubernetes.io/projected/ca21c26b-4510-498d-819f-2bc6fa4a2dae-kube-api-access-km9xq\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:44.252889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:44.252781    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn" event={"ID":"ca21c26b-4510-498d-819f-2bc6fa4a2dae","Type":"ContainerDied","Data":"6ebfd96e0a809ea0977ec3448eafaa273f3bfe64153c86e8c03a6a595f8c292f"}
Sep 08 14:15:44.252889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:44.252838    2564 scope.go:117] "RemoveContainer" containerID="e91b2138d980ff667a7252148d1734d5237864ded03c880e1a3168a595c67c7a"
Sep 08 14:15:44.252889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:44.252788    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"
Sep 08 14:15:44.273681 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:44.273653    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"]
Sep 08 14:15:44.280583 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:44.280541    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-cb068d-openshift-default-58b9dc7c5b-t5bxn"]
Sep 08 14:15:45.959808 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:45.959764    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" path="/var/lib/kubelet/pods/ca21c26b-4510-498d-819f-2bc6fa4a2dae/volumes"
Sep 08 14:15:49.109402 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.109372    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr"]
Sep 08 14:15:49.113790 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.113752    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-94eadd-29814615-qvhmr"]
Sep 08 14:15:49.140815 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.140772    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"]
Sep 08 14:15:49.141100 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.141071    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj" podUID="77b9b8ff-c7b4-482a-953f-a8211f69e50c" containerName="payload-processing" containerID="cri-o://59757cf7d6da6299fbe6eb5c1affea78954ba7120058860ed46d25cbdff8caca" gracePeriod=30
Sep 08 14:15:49.285242 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.285203    2564 generic.go:356] "Generic (PLEG): container finished" podID="77b9b8ff-c7b4-482a-953f-a8211f69e50c" containerID="59757cf7d6da6299fbe6eb5c1affea78954ba7120058860ed46d25cbdff8caca" exitCode=0
Sep 08 14:15:49.285427 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.285280    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj" event={"ID":"77b9b8ff-c7b4-482a-953f-a8211f69e50c","Type":"ContainerDied","Data":"59757cf7d6da6299fbe6eb5c1affea78954ba7120058860ed46d25cbdff8caca"}
Sep 08 14:15:49.403101 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.403071    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"]
Sep 08 14:15:49.409736 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.409711    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-94eadd-gd4xc"]
Sep 08 14:15:49.413878 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.413840    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:15:49.519512 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.519473    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5j2q\" (UniqueName: \"kubernetes.io/projected/77b9b8ff-c7b4-482a-953f-a8211f69e50c-kube-api-access-n5j2q\") pod \"77b9b8ff-c7b4-482a-953f-a8211f69e50c\" (UID: \"77b9b8ff-c7b4-482a-953f-a8211f69e50c\") "
Sep 08 14:15:49.519702 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.519606    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77b9b8ff-c7b4-482a-953f-a8211f69e50c-plugins-config-volume\") pod \"77b9b8ff-c7b4-482a-953f-a8211f69e50c\" (UID: \"77b9b8ff-c7b4-482a-953f-a8211f69e50c\") "
Sep 08 14:15:49.520010 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.519985    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77b9b8ff-c7b4-482a-953f-a8211f69e50c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "77b9b8ff-c7b4-482a-953f-a8211f69e50c" (UID: "77b9b8ff-c7b4-482a-953f-a8211f69e50c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:15:49.521916 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.521890    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77b9b8ff-c7b4-482a-953f-a8211f69e50c-kube-api-access-n5j2q" (OuterVolumeSpecName: "kube-api-access-n5j2q") pod "77b9b8ff-c7b4-482a-953f-a8211f69e50c" (UID: "77b9b8ff-c7b4-482a-953f-a8211f69e50c"). InnerVolumeSpecName "kube-api-access-n5j2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:15:49.620621 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.620591    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5j2q\" (UniqueName: \"kubernetes.io/projected/77b9b8ff-c7b4-482a-953f-a8211f69e50c-kube-api-access-n5j2q\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:49.620621 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.620620    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77b9b8ff-c7b4-482a-953f-a8211f69e50c-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:15:49.960587 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.960558    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="869e083c-e354-4050-92f7-9129ac7ca466" path="/var/lib/kubelet/pods/869e083c-e354-4050-92f7-9129ac7ca466/volumes"
Sep 08 14:15:49.960852 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:49.960840    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8" path="/var/lib/kubelet/pods/aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8/volumes"
Sep 08 14:15:50.308652 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:50.308609    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj" event={"ID":"77b9b8ff-c7b4-482a-953f-a8211f69e50c","Type":"ContainerDied","Data":"4defcad23a24eaa6104e07dd28713a399d2375a7fbac51e601fbfccddaa26046"}
Sep 08 14:15:50.309129 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:50.308672    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"
Sep 08 14:15:50.309129 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:50.308716    2564 scope.go:117] "RemoveContainer" containerID="59757cf7d6da6299fbe6eb5c1affea78954ba7120058860ed46d25cbdff8caca"
Sep 08 14:15:50.329464 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:50.329432    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"]
Sep 08 14:15:50.333646 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:50.333621    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-94eadd-7b99f68df-snskj"]
Sep 08 14:15:51.316987 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:51.316946    2564 generic.go:356] "Generic (PLEG): container finished" podID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerID="8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c" exitCode=0
Sep 08 14:15:51.317459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:51.317018    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" event={"ID":"0ee5f985-7473-447b-a1f3-d886935ae0bc","Type":"ContainerDied","Data":"8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c"}
Sep 08 14:15:51.317459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:51.317395    2564 scope.go:117] "RemoveContainer" containerID="8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c"
Sep 08 14:15:51.318480 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:51.318448    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:15:51.318576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:51.318520    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:15:51.318576 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:51.318556    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:15:51.959718 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:51.959686    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b9b8ff-c7b4-482a-953f-a8211f69e50c" path="/var/lib/kubelet/pods/77b9b8ff-c7b4-482a-953f-a8211f69e50c/volumes"
Sep 08 14:15:52.324718 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:52.324685    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" event={"ID":"0ee5f985-7473-447b-a1f3-d886935ae0bc","Type":"ContainerStarted","Data":"b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e"}
Sep 08 14:15:52.393993 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:52.393957    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:52.395323 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:52.395293    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 14:15:52.395436 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:52.395348    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 14:15:53.394013 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:53.393980    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 14:15:53.394405 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:53.394034    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 14:15:54.393507 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:54.393461    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 14:15:54.393701 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:54.393525    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 14:15:55.394368 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:55.394326    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 14:15:55.394848 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:55.394400    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 14:15:56.393827 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:56.393790    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 14:15:56.394119 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:56.393850    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 14:15:56.489322 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:56.489292    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"]
Sep 08 14:15:56.489736 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:56.489509    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy" containerID="cri-o://3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc" gracePeriod=30
Sep 08 14:15:57.111349 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:57.111309    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:57.393839 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:57.393757    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 14:15:57.393839 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:57.393823    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 14:15:58.394026 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:58.393987    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 08 14:15:58.394442 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:58.394050    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 08 14:15:59.398321 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:59.398293    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:15:59.399271 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:15:59.399249    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:16:01.745060 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.745031    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:16:01.842785 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.842695    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-token\") pod \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") "
Sep 08 14:16:01.842785 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.842740    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6rkh\" (UniqueName: \"kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-kube-api-access-v6rkh\") pod \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") "
Sep 08 14:16:01.843037 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.842787    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-socket\") pod \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") "
Sep 08 14:16:01.843037 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.842925    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-envoy\") pod \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") "
Sep 08 14:16:01.843037 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843016    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-data\") pod \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") "
Sep 08 14:16:01.843174 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843047    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-podinfo\") pod \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") "
Sep 08 14:16:01.843174 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843049    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" (UID: "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:16:01.843174 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843093    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-credential-socket\") pod \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") "
Sep 08 14:16:01.843174 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843136    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-certs\") pod \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") "
Sep 08 14:16:01.843388 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843173    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istiod-ca-cert\") pod \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\" (UID: \"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247\") "
Sep 08 14:16:01.843388 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843270    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-data" (OuterVolumeSpecName: "istio-data") pod "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" (UID: "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:16:01.843388 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843373    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" (UID: "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:16:01.843557 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843400    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" (UID: "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:16:01.843619 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843567    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:16:01.843619 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843582    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-data\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:16:01.843619 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843596    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-credential-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:16:01.843619 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843610    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-workload-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:16:01.844025 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.843786    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" (UID: "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:16:01.845203 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.845180    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-token" (OuterVolumeSpecName: "istio-token") pod "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" (UID: "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:16:01.845493 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.845470    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-kube-api-access-v6rkh" (OuterVolumeSpecName: "kube-api-access-v6rkh") pod "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" (UID: "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247"). InnerVolumeSpecName "kube-api-access-v6rkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:16:01.845741 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.845723    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" (UID: "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:16:01.845817 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.845735    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" (UID: "8ed6dea7-cccc-4bfd-a9ca-72346ae7d247"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:16:01.945129 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.945087    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-token\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:16:01.945129 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.945123    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6rkh\" (UniqueName: \"kubernetes.io/projected/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-kube-api-access-v6rkh\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:16:01.945129 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.945138    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-envoy\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:16:01.945379 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.945150    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istio-podinfo\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:16:01.945379 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:01.945161    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247-istiod-ca-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:16:02.390596 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.390555    2564 generic.go:356] "Generic (PLEG): container finished" podID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerID="3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc" exitCode=0
Sep 08 14:16:02.390806 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.390614    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"
Sep 08 14:16:02.390806 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.390640    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" event={"ID":"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247","Type":"ContainerDied","Data":"3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc"}
Sep 08 14:16:02.390806 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.390725    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w" event={"ID":"8ed6dea7-cccc-4bfd-a9ca-72346ae7d247","Type":"ContainerDied","Data":"41219db537975790c9ac51caf663c6eeaff97148844756efb52945b25332c7a1"}
Sep 08 14:16:02.390806 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.390749    2564 scope.go:117] "RemoveContainer" containerID="3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc"
Sep 08 14:16:02.399991 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.399972    2564 scope.go:117] "RemoveContainer" containerID="e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743"
Sep 08 14:16:02.408436 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.408414    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"]
Sep 08 14:16:02.408773 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.408746    2564 scope.go:117] "RemoveContainer" containerID="3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc"
Sep 08 14:16:02.409072 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:16:02.409053    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc\": container with ID starting with 3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc not found: ID does not exist" containerID="3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc"
Sep 08 14:16:02.409148 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.409082    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc"} err="failed to get container status \"3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc\": rpc error: code = NotFound desc = could not find container \"3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc\": container with ID starting with 3e9e3aaadac382f0468f03d1e7972b46c42e7298dad8c5af3b383affe8e5babc not found: ID does not exist"
Sep 08 14:16:02.409148 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.409100    2564 scope.go:117] "RemoveContainer" containerID="e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743"
Sep 08 14:16:02.409335 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:16:02.409321    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743\": container with ID starting with e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743 not found: ID does not exist" containerID="e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743"
Sep 08 14:16:02.409378 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.409338    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743"} err="failed to get container status \"e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743\": rpc error: code = NotFound desc = could not find container \"e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743\": container with ID starting with e391851df26082a4e31846ef4c64d23f7c354fa8886b07266b294ff53f688743 not found: ID does not exist"
Sep 08 14:16:02.412438 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:02.412416    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-94eadd-openshift-default-775cfb4746-qzr5w"]
Sep 08 14:16:03.958933 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:03.958904    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" path="/var/lib/kubelet/pods/8ed6dea7-cccc-4bfd-a9ca-72346ae7d247/volumes"
Sep 08 14:16:08.480174 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480134    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"]
Sep 08 14:16:08.480632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480552    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy"
Sep 08 14:16:08.480632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480565    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy"
Sep 08 14:16:08.480632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480580    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy"
Sep 08 14:16:08.480632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480586    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy"
Sep 08 14:16:08.480632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480594    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b9b8ff-c7b4-482a-953f-a8211f69e50c" containerName="payload-processing"
Sep 08 14:16:08.480632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480599    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b9b8ff-c7b4-482a-953f-a8211f69e50c" containerName="payload-processing"
Sep 08 14:16:08.480632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480619    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy"
Sep 08 14:16:08.480632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480627    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy"
Sep 08 14:16:08.480632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480636    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy"
Sep 08 14:16:08.481178 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480644    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy"
Sep 08 14:16:08.481178 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480670    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8" containerName="revoke-keys"
Sep 08 14:16:08.481178 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480678    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8" containerName="revoke-keys"
Sep 08 14:16:08.481178 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480767    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy"
Sep 08 14:16:08.481178 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480775    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ed6dea7-cccc-4bfd-a9ca-72346ae7d247" containerName="istio-proxy"
Sep 08 14:16:08.481178 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480781    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="aabdfb7a-d6f5-4c94-b6c9-6d3c586338b8" containerName="revoke-keys"
Sep 08 14:16:08.481178 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480789    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy"
Sep 08 14:16:08.481178 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480797    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b9b8ff-c7b4-482a-953f-a8211f69e50c" containerName="payload-processing"
Sep 08 14:16:08.481178 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.480807    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca21c26b-4510-498d-819f-2bc6fa4a2dae" containerName="istio-proxy"
Sep 08 14:16:08.483893 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.483858    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.487158 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.487135    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ec7c379e-openshift-default-dockercfg-mqchn\""
Sep 08 14:16:08.499927 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.499902    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"]
Sep 08 14:16:08.608537 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.608449    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2t9q\" (UniqueName: \"kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-kube-api-access-j2t9q\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.608537 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.608506    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-socket\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.608765 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.608568    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-data\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.608765 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.608616    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf90a7d8-063d-4b9d-9b30-282c99b95109-istiod-ca-cert\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.608765 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.608644    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-certs\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.608765 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.608668    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-credential-socket\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.608765 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.608719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-podinfo\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.608765 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.608750    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-token\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.609029 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.608770    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-envoy\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.710343 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.710305    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-token\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.710343 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.710354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-envoy\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.710594 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.710439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2t9q\" (UniqueName: \"kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-kube-api-access-j2t9q\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.710594 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.710502    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-socket\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.710594 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.710538    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-data\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.710594 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.710570    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf90a7d8-063d-4b9d-9b30-282c99b95109-istiod-ca-cert\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.710848 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.710600    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-certs\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.710848 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.710629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-credential-socket\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.710848 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.710679    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-podinfo\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.711089 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.711031    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-data\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.711089 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.711052    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-certs\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.711463 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.711229    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-credential-socket\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.711463 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.711320    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf90a7d8-063d-4b9d-9b30-282c99b95109-istiod-ca-cert\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.711579 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.711476    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-socket\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.713061 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.713042    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-envoy\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.713172 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.713157    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-podinfo\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.719398 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.719372    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-token\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.719525 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.719508    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2t9q\" (UniqueName: \"kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-kube-api-access-j2t9q\") pod \"e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") " pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:08.795775 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:08.795736    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:09.076416 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:09.076387    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"]
Sep 08 14:16:09.078272 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:16:09.078238    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf90a7d8_063d_4b9d_9b30_282c99b95109.slice/crio-641c093dfdbabec86fcf2be72bdc8950a922e3c751ed75f3c5558ca760fedcac WatchSource:0}: Error finding container 641c093dfdbabec86fcf2be72bdc8950a922e3c751ed75f3c5558ca760fedcac: Status 404 returned error can't find the container with id 641c093dfdbabec86fcf2be72bdc8950a922e3c751ed75f3c5558ca760fedcac
Sep 08 14:16:09.081224 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:09.081181    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:16:09.081313 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:09.081271    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:16:09.081362 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:09.081338    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:16:09.434381 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:09.434288    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" event={"ID":"cf90a7d8-063d-4b9d-9b30-282c99b95109","Type":"ContainerStarted","Data":"3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899"}
Sep 08 14:16:09.434381 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:09.434345    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" event={"ID":"cf90a7d8-063d-4b9d-9b30-282c99b95109","Type":"ContainerStarted","Data":"641c093dfdbabec86fcf2be72bdc8950a922e3c751ed75f3c5558ca760fedcac"}
Sep 08 14:16:09.459631 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:09.459576    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" podStartSLOduration=1.459559611 podStartE2EDuration="1.459559611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:16:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:16:09.458084605 +0000 UTC m=+1059.953055294" watchObservedRunningTime="2026-09-08 14:16:09.459559611 +0000 UTC m=+1059.954530364"
Sep 08 14:16:09.796623 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:09.796586    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:10.801606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:10.801571    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:11.451469 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:11.451436    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:11.452322 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:11.452303    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:30.076950 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:30.076916    2564 scope.go:117] "RemoveContainer" containerID="98f0fb267a1d2c6224272354a834870bd1c2673017b0e98d27733f089037a552"
Sep 08 14:16:32.577995 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:32.577962    2564 generic.go:356] "Generic (PLEG): container finished" podID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerID="3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899" exitCode=0
Sep 08 14:16:32.578414 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:32.578019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" event={"ID":"cf90a7d8-063d-4b9d-9b30-282c99b95109","Type":"ContainerDied","Data":"3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899"}
Sep 08 14:16:32.578461 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:32.578425    2564 scope.go:117] "RemoveContainer" containerID="3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899"
Sep 08 14:16:32.579495 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:32.579463    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:16:32.579600 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:32.579561    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:16:32.579669 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:32.579616    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:16:33.586087 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:33.586050    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" event={"ID":"cf90a7d8-063d-4b9d-9b30-282c99b95109","Type":"ContainerStarted","Data":"688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf"}
Sep 08 14:16:33.795986 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:33.795946    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:33.796213 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:33.796187    2564 patch_prober.go:28] interesting pod/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 08 14:16:33.796301 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:33.796248    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 08 14:16:34.796273 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:34.796233    2564 patch_prober.go:28] interesting pod/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 08 14:16:34.796665 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:34.796288    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 08 14:16:35.797118 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:35.797068    2564 patch_prober.go:28] interesting pod/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 08 14:16:35.797512 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:35.797154    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 08 14:16:36.796758 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:36.796722    2564 patch_prober.go:28] interesting pod/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 08 14:16:36.796998 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:36.796791    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 08 14:16:37.796944 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:37.796898    2564 patch_prober.go:28] interesting pod/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 08 14:16:37.797343 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:37.796980    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 08 14:16:38.796811 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:38.796767    2564 patch_prober.go:28] interesting pod/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 08 14:16:38.797040 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:38.796847    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 08 14:16:39.800934 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:39.800897    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:39.801315 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:39.801270    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:16:39.801903 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:16:39.801885    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:17:30.114922 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:17:30.114884    2564 scope.go:117] "RemoveContainer" containerID="b25ce01366e833d18d49e640f6680833f1d4c2a609248729cfcaea7f6b208855"
Sep 08 14:17:30.123359 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:17:30.123339    2564 scope.go:117] "RemoveContainer" containerID="a3bf6df12709c932644572ae9d0f31fa20ab3bd805004c057c66bdcecc2721f2"
Sep 08 14:18:30.146702 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:18:30.146667    2564 scope.go:117] "RemoveContainer" containerID="b9f5ff83ba00140ec574498ea594cb0afe296235d37b455b66428c72d26d4841"
Sep 08 14:18:30.155305 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:18:30.155285    2564 scope.go:117] "RemoveContainer" containerID="202006ebfc1050e04a2c3ad5df16c0fd8b4b2000e23093d747aa0d857036ab41"
Sep 08 14:18:33.865939 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:18:33.865908    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:18:33.873458 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:18:33.873437    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:19:52.972925 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:52.972817    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"]
Sep 08 14:19:52.976003 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:52.975978    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"
Sep 08 14:19:52.979152 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:52.979135    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:19:52.980346 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:52.980321    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:19:53.029315 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:53.029291    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"]
Sep 08 14:19:53.072497 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:53.072468    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e716b2e6-d394-4321-baa6-975b763c7554-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ec7c379e-przfd\" (UID: \"e716b2e6-d394-4321-baa6-975b763c7554\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"
Sep 08 14:19:53.173383 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:53.173347    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e716b2e6-d394-4321-baa6-975b763c7554-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ec7c379e-przfd\" (UID: \"e716b2e6-d394-4321-baa6-975b763c7554\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"
Sep 08 14:19:53.174059 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:53.174039    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e716b2e6-d394-4321-baa6-975b763c7554-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ec7c379e-przfd\" (UID: \"e716b2e6-d394-4321-baa6-975b763c7554\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"
Sep 08 14:19:53.286046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:53.285968    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"
Sep 08 14:19:53.559181 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:53.559150    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"]
Sep 08 14:19:53.561324 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:19:53.561299    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode716b2e6_d394_4321_baa6_975b763c7554.slice/crio-e17d265cac8c086d386e76a035006368e4319381d687d38ffbea0244aed79770 WatchSource:0}: Error finding container e17d265cac8c086d386e76a035006368e4319381d687d38ffbea0244aed79770: Status 404 returned error can't find the container with id e17d265cac8c086d386e76a035006368e4319381d687d38ffbea0244aed79770
Sep 08 14:19:53.647435 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:53.647402    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd" event={"ID":"e716b2e6-d394-4321-baa6-975b763c7554","Type":"ContainerStarted","Data":"e17d265cac8c086d386e76a035006368e4319381d687d38ffbea0244aed79770"}
Sep 08 14:19:54.655794 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:54.655764    2564 generic.go:356] "Generic (PLEG): container finished" podID="e716b2e6-d394-4321-baa6-975b763c7554" containerID="a1919244399668da86574643904b5d3b5a25a251ee74ac54d0bd8ef0ba9c4e69" exitCode=0
Sep 08 14:19:54.656305 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:54.655851    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd" event={"ID":"e716b2e6-d394-4321-baa6-975b763c7554","Type":"ContainerDied","Data":"a1919244399668da86574643904b5d3b5a25a251ee74ac54d0bd8ef0ba9c4e69"}
Sep 08 14:19:55.792591 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:55.792565    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"
Sep 08 14:19:55.898380 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:55.898331    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e716b2e6-d394-4321-baa6-975b763c7554-maas-api-service-ca\") pod \"e716b2e6-d394-4321-baa6-975b763c7554\" (UID: \"e716b2e6-d394-4321-baa6-975b763c7554\") "
Sep 08 14:19:55.898713 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:55.898690    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e716b2e6-d394-4321-baa6-975b763c7554-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e716b2e6-d394-4321-baa6-975b763c7554" (UID: "e716b2e6-d394-4321-baa6-975b763c7554"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:19:55.999442 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:55.999417    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e716b2e6-d394-4321-baa6-975b763c7554-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:19:56.667416 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:56.667388    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"
Sep 08 14:19:56.667608 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:56.667385    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd" event={"ID":"e716b2e6-d394-4321-baa6-975b763c7554","Type":"ContainerDied","Data":"e17d265cac8c086d386e76a035006368e4319381d687d38ffbea0244aed79770"}
Sep 08 14:19:56.667608 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:19:56.667525    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e17d265cac8c086d386e76a035006368e4319381d687d38ffbea0244aed79770"
Sep 08 14:20:03.937822 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:03.937786    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"]
Sep 08 14:20:03.940363 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:03.940337    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ec7c379e-przfd"]
Sep 08 14:20:03.958466 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:03.958428    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e716b2e6-d394-4321-baa6-975b763c7554" path="/var/lib/kubelet/pods/e716b2e6-d394-4321-baa6-975b763c7554/volumes"
Sep 08 14:20:30.186515 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:30.186484    2564 scope.go:117] "RemoveContainer" containerID="f968fb475199ebccb65c35b054b660a974801308c55adb001a4400be46383691"
Sep 08 14:20:30.195532 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:30.195511    2564 scope.go:117] "RemoveContainer" containerID="570291283a8890dcf1a6c599852b421403435ca9ff374da7df15ba3b222a6786"
Sep 08 14:20:53.292416 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:53.292334    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"]
Sep 08 14:20:53.292822 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:53.292568    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy" containerID="cri-o://688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf" gracePeriod=30
Sep 08 14:20:54.803296 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:54.803260    2564 patch_prober.go:28] interesting pod/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 14:20:54.803672 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:54.803352    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 14:20:58.534296 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.534269    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:20:58.611859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.611769    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-certs\") pod \"cf90a7d8-063d-4b9d-9b30-282c99b95109\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") "
Sep 08 14:20:58.611859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.611807    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf90a7d8-063d-4b9d-9b30-282c99b95109-istiod-ca-cert\") pod \"cf90a7d8-063d-4b9d-9b30-282c99b95109\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") "
Sep 08 14:20:58.611859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.611824    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-podinfo\") pod \"cf90a7d8-063d-4b9d-9b30-282c99b95109\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") "
Sep 08 14:20:58.611859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.611841    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-credential-socket\") pod \"cf90a7d8-063d-4b9d-9b30-282c99b95109\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") "
Sep 08 14:20:58.611859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.611879    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-data\") pod \"cf90a7d8-063d-4b9d-9b30-282c99b95109\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") "
Sep 08 14:20:58.612274 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612025    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-envoy\") pod \"cf90a7d8-063d-4b9d-9b30-282c99b95109\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") "
Sep 08 14:20:58.612274 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612102    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-socket\") pod \"cf90a7d8-063d-4b9d-9b30-282c99b95109\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") "
Sep 08 14:20:58.612274 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612168    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf90a7d8-063d-4b9d-9b30-282c99b95109" (UID: "cf90a7d8-063d-4b9d-9b30-282c99b95109"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:20:58.612274 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612182    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-token\") pod \"cf90a7d8-063d-4b9d-9b30-282c99b95109\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") "
Sep 08 14:20:58.612274 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612187    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf90a7d8-063d-4b9d-9b30-282c99b95109" (UID: "cf90a7d8-063d-4b9d-9b30-282c99b95109"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:20:58.612274 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612202    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf90a7d8-063d-4b9d-9b30-282c99b95109" (UID: "cf90a7d8-063d-4b9d-9b30-282c99b95109"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:20:58.612274 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612242    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2t9q\" (UniqueName: \"kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-kube-api-access-j2t9q\") pod \"cf90a7d8-063d-4b9d-9b30-282c99b95109\" (UID: \"cf90a7d8-063d-4b9d-9b30-282c99b95109\") "
Sep 08 14:20:58.612614 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612283    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf90a7d8-063d-4b9d-9b30-282c99b95109-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf90a7d8-063d-4b9d-9b30-282c99b95109" (UID: "cf90a7d8-063d-4b9d-9b30-282c99b95109"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:20:58.612614 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612392    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf90a7d8-063d-4b9d-9b30-282c99b95109" (UID: "cf90a7d8-063d-4b9d-9b30-282c99b95109"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:20:58.612614 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612610    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:20:58.612882 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612628    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf90a7d8-063d-4b9d-9b30-282c99b95109-istiod-ca-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:20:58.612882 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612642    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-credential-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:20:58.612882 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612657    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-data\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:20:58.612882 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.612669    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-workload-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:20:58.614365 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.614333    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf90a7d8-063d-4b9d-9b30-282c99b95109" (UID: "cf90a7d8-063d-4b9d-9b30-282c99b95109"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:20:58.614365 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.614344    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf90a7d8-063d-4b9d-9b30-282c99b95109" (UID: "cf90a7d8-063d-4b9d-9b30-282c99b95109"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:20:58.614521 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.614430    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf90a7d8-063d-4b9d-9b30-282c99b95109" (UID: "cf90a7d8-063d-4b9d-9b30-282c99b95109"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:20:58.614521 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.614442    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-kube-api-access-j2t9q" (OuterVolumeSpecName: "kube-api-access-j2t9q") pod "cf90a7d8-063d-4b9d-9b30-282c99b95109" (UID: "cf90a7d8-063d-4b9d-9b30-282c99b95109"). InnerVolumeSpecName "kube-api-access-j2t9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:20:58.713851 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.713819    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-token\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:20:58.713851 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.713848    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2t9q\" (UniqueName: \"kubernetes.io/projected/cf90a7d8-063d-4b9d-9b30-282c99b95109-kube-api-access-j2t9q\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:20:58.713851 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.713883    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-podinfo\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:20:58.713851 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.713892    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf90a7d8-063d-4b9d-9b30-282c99b95109-istio-envoy\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:20:58.997212 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.997173    2564 generic.go:356] "Generic (PLEG): container finished" podID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerID="688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf" exitCode=0
Sep 08 14:20:58.997420 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.997237    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"
Sep 08 14:20:58.997420 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.997262    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" event={"ID":"cf90a7d8-063d-4b9d-9b30-282c99b95109","Type":"ContainerDied","Data":"688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf"}
Sep 08 14:20:58.997420 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.997327    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8" event={"ID":"cf90a7d8-063d-4b9d-9b30-282c99b95109","Type":"ContainerDied","Data":"641c093dfdbabec86fcf2be72bdc8950a922e3c751ed75f3c5558ca760fedcac"}
Sep 08 14:20:58.997420 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:58.997356    2564 scope.go:117] "RemoveContainer" containerID="688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf"
Sep 08 14:20:59.007550 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:59.007533    2564 scope.go:117] "RemoveContainer" containerID="3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899"
Sep 08 14:20:59.015892 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:59.015851    2564 scope.go:117] "RemoveContainer" containerID="688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf"
Sep 08 14:20:59.016148 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:20:59.016129    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf\": container with ID starting with 688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf not found: ID does not exist" containerID="688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf"
Sep 08 14:20:59.016229 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:59.016157    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf"} err="failed to get container status \"688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf\": rpc error: code = NotFound desc = could not find container \"688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf\": container with ID starting with 688b0fb8b75a6c386bfd836f988915ef7a254052f35cb964baad999381c53fdf not found: ID does not exist"
Sep 08 14:20:59.016229 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:59.016172    2564 scope.go:117] "RemoveContainer" containerID="3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899"
Sep 08 14:20:59.016396 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:20:59.016380    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899\": container with ID starting with 3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899 not found: ID does not exist" containerID="3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899"
Sep 08 14:20:59.016454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:59.016401    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899"} err="failed to get container status \"3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899\": rpc error: code = NotFound desc = could not find container \"3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899\": container with ID starting with 3deae2f27c1e843b71a4b68c0939caab1bec8bb6b8f4077c668dd7ea26e0b899 not found: ID does not exist"
Sep 08 14:20:59.021048 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:59.021028    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"]
Sep 08 14:20:59.025799 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:59.025778    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ec7c379e-openshift-default-55dcb4b844-bqqr8"]
Sep 08 14:20:59.958397 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:20:59.958361    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" path="/var/lib/kubelet/pods/cf90a7d8-063d-4b9d-9b30-282c99b95109/volumes"
Sep 08 14:21:30.226307 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:30.226279    2564 scope.go:117] "RemoveContainer" containerID="d63558a431f6a9e1737801cd0d06210121d57fb084e5626eea5c4f333574ba71"
Sep 08 14:21:30.234876 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:30.234839    2564 scope.go:117] "RemoveContainer" containerID="8aa82ec3492f47813eab84aa962c5d9e3b956e16eaf1f491067db011f1cc902d"
Sep 08 14:21:30.242631 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:30.242611    2564 scope.go:117] "RemoveContainer" containerID="51369b6276bfed39b59193ca7e42e9f93f53a7c85446136f9c137aa7bf0533a8"
Sep 08 14:21:49.191833 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.191793    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"]
Sep 08 14:21:49.194291 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.192350    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy"
Sep 08 14:21:49.194291 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.192366    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy"
Sep 08 14:21:49.194291 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.192398    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e716b2e6-d394-4321-baa6-975b763c7554" containerName="revoke-keys"
Sep 08 14:21:49.194291 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.192403    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e716b2e6-d394-4321-baa6-975b763c7554" containerName="revoke-keys"
Sep 08 14:21:49.194291 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.192458    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e716b2e6-d394-4321-baa6-975b763c7554" containerName="revoke-keys"
Sep 08 14:21:49.194291 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.192465    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy"
Sep 08 14:21:49.195639 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.195620    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.200133 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.200100    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a8b757-openshift-default-dockercfg-76hs7\""
Sep 08 14:21:49.226978 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.226948    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"]
Sep 08 14:21:49.293546 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.293516    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-istio-token\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.293725 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.293575    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-certs\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.293725 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.293680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-data\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.293828 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.293726    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-socket\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.293828 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.293755    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e8781c7-053e-4317-b816-fc23d9c000da-istiod-ca-cert\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.293828 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.293776    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-envoy\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.293828 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.293795    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-credential-socket\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.294031 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.293839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8rhc\" (UniqueName: \"kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-kube-api-access-h8rhc\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.294031 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.293891    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e8781c7-053e-4317-b816-fc23d9c000da-istio-podinfo\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.395383 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.395349    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8rhc\" (UniqueName: \"kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-kube-api-access-h8rhc\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.395570 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.395399    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e8781c7-053e-4317-b816-fc23d9c000da-istio-podinfo\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.395570 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.395448    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-istio-token\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.395570 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.395493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-certs\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.395570 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.395546    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-data\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.395801 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.395578    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-socket\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.395801 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.395613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e8781c7-053e-4317-b816-fc23d9c000da-istiod-ca-cert\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.395801 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.395643    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-envoy\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.395801 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.395670    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-credential-socket\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.396097 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.396072    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-certs\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.396291 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.396260    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-socket\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.396456 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.396438    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e8781c7-053e-4317-b816-fc23d9c000da-istiod-ca-cert\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.396518 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.396498    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-credential-socket\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.396601 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.396575    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-data\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.397961 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.397941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-envoy\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.398177 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.398161    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e8781c7-053e-4317-b816-fc23d9c000da-istio-podinfo\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.407477 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.407454    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-istio-token\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.407568 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.407476    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8rhc\" (UniqueName: \"kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-kube-api-access-h8rhc\") pod \"e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") " pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.506477 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.506452    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:49.790093 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.790070    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"]
Sep 08 14:21:49.792845 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:21:49.792815    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e8781c7_053e_4317_b816_fc23d9c000da.slice/crio-649679c8af69e89bd67e396e0a3c9fa45131bfc3dc7841af052ac3ae6ca0a33a WatchSource:0}: Error finding container 649679c8af69e89bd67e396e0a3c9fa45131bfc3dc7841af052ac3ae6ca0a33a: Status 404 returned error can't find the container with id 649679c8af69e89bd67e396e0a3c9fa45131bfc3dc7841af052ac3ae6ca0a33a
Sep 08 14:21:49.794999 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.794966    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:21:49.795102 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.795045    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:21:49.795102 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:49.795091    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:21:50.253623 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:50.253592    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" event={"ID":"4e8781c7-053e-4317-b816-fc23d9c000da","Type":"ContainerStarted","Data":"5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5"}
Sep 08 14:21:50.254045 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:50.253666    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" event={"ID":"4e8781c7-053e-4317-b816-fc23d9c000da","Type":"ContainerStarted","Data":"649679c8af69e89bd67e396e0a3c9fa45131bfc3dc7841af052ac3ae6ca0a33a"}
Sep 08 14:21:50.285741 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:50.285347    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" podStartSLOduration=1.285331611 podStartE2EDuration="1.285331611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:21:50.284507558 +0000 UTC m=+1400.779478242" watchObservedRunningTime="2026-09-08 14:21:50.285331611 +0000 UTC m=+1400.780302294"
Sep 08 14:21:50.506721 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:50.506640    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:50.512093 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:50.512068    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:51.258839 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:51.258804    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:21:51.259684 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:21:51.259660    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:22:13.377990 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:13.377903    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e8781c7-053e-4317-b816-fc23d9c000da" containerID="5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5" exitCode=0
Sep 08 14:22:13.377990 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:13.377967    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" event={"ID":"4e8781c7-053e-4317-b816-fc23d9c000da","Type":"ContainerDied","Data":"5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5"}
Sep 08 14:22:13.378415 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:13.378358    2564 scope.go:117] "RemoveContainer" containerID="5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5"
Sep 08 14:22:13.379459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:13.379429    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:22:13.379516 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:13.379506    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:22:13.379559 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:13.379545    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:22:14.384681 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:14.384642    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" event={"ID":"4e8781c7-053e-4317-b816-fc23d9c000da","Type":"ContainerStarted","Data":"7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5"}
Sep 08 14:22:14.507373 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:14.507339    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:22:14.507580 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:14.507558    2564 patch_prober.go:28] interesting pod/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 08 14:22:14.507654 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:14.507624    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 08 14:22:15.507785 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:15.507749    2564 patch_prober.go:28] interesting pod/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 08 14:22:15.508246 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:15.507818    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 08 14:22:16.507623 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:16.507587    2564 patch_prober.go:28] interesting pod/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 08 14:22:16.507807 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:16.507664    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 08 14:22:17.507170 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:17.507119    2564 patch_prober.go:28] interesting pod/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 08 14:22:17.507393 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:17.507180    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 08 14:22:18.089319 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.089289    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"]
Sep 08 14:22:18.089727 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.089708    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy"
Sep 08 14:22:18.089727 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.089723    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy"
Sep 08 14:22:18.089893 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.089825    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf90a7d8-063d-4b9d-9b30-282c99b95109" containerName="istio-proxy"
Sep 08 14:22:18.092140 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.092123    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.095101 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.095079    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-2310c4-openshift-default-dockercfg-p97sz\""
Sep 08 14:22:18.107172 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.107150    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"]
Sep 08 14:22:18.184366 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.184329    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-data\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.184366 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.184364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-token\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.184680 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.184401    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istiod-ca-cert\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.184680 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.184487    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-envoy\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.184680 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.184527    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-podinfo\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.184680 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.184585    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-socket\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.184942 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.184780    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-credential-socket\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.184942 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.184909    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-certs\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.185074 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.184969    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pllm8\" (UniqueName: \"kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-kube-api-access-pllm8\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.286314 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286283    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-socket\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.286493 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286326    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-credential-socket\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.286493 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286368    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-certs\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.286493 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pllm8\" (UniqueName: \"kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-kube-api-access-pllm8\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.286606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286559    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-data\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.286662 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-token\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.286662 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286646    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istiod-ca-cert\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.286793 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286700    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-envoy\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.286793 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286721    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-socket\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.286793 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286728    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-podinfo\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.286793 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286777    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-credential-socket\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.287047 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286830    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-certs\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.287047 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.286941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-data\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.287521 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.287467    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istiod-ca-cert\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.289070 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.289049    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-envoy\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.289503 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.289481    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-podinfo\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.295810 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.295786    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pllm8\" (UniqueName: \"kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-kube-api-access-pllm8\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.297606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.297584    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-token\") pod \"e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") " pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.403154 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.403061    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:18.507571 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.507482    2564 patch_prober.go:28] interesting pod/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 08 14:22:18.507571 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.507557    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 08 14:22:18.706321 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.706293    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"]
Sep 08 14:22:18.709088 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:22:18.709051    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8d17bf9_6491_4504_8cfd_e3f9eba2f417.slice/crio-7c2fab1a15afdec663f0268d31b7c3f249baac2a7aedf923b1caaa7e3b6a64c4 WatchSource:0}: Error finding container 7c2fab1a15afdec663f0268d31b7c3f249baac2a7aedf923b1caaa7e3b6a64c4: Status 404 returned error can't find the container with id 7c2fab1a15afdec663f0268d31b7c3f249baac2a7aedf923b1caaa7e3b6a64c4
Sep 08 14:22:18.716889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.712473    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:22:18.716889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.712624    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:22:18.716889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:18.712720    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:22:19.410626 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:19.410596    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" event={"ID":"d8d17bf9-6491-4504-8cfd-e3f9eba2f417","Type":"ContainerStarted","Data":"b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079"}
Sep 08 14:22:19.411047 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:19.410685    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" event={"ID":"d8d17bf9-6491-4504-8cfd-e3f9eba2f417","Type":"ContainerStarted","Data":"7c2fab1a15afdec663f0268d31b7c3f249baac2a7aedf923b1caaa7e3b6a64c4"}
Sep 08 14:22:19.434729 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:19.434664    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" podStartSLOduration=1.434644225 podStartE2EDuration="1.434644225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:22:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:22:19.431959614 +0000 UTC m=+1429.926930611" watchObservedRunningTime="2026-09-08 14:22:19.434644225 +0000 UTC m=+1429.929614909"
Sep 08 14:22:19.507072 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:19.507043    2564 patch_prober.go:28] interesting pod/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 08 14:22:19.507237 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:19.507096    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 08 14:22:20.404250 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:20.404216    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:20.409426 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:20.409404    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:20.415989 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:20.415958    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:20.416884 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:20.416842    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:20.512300 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:20.512274    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:22:20.512632 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:20.512613    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:22:20.513405 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:20.513386    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:22:23.267263 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.267232    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"]
Sep 08 14:22:23.270272 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.270244    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:22:23.272990 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.272967    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-2310c4\""
Sep 08 14:22:23.273201 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.273177    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-2310c4-dockercfg-wwvk8\""
Sep 08 14:22:23.282295 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.282274    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"]
Sep 08 14:22:23.340579 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.340549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15ce599e-f880-46d7-91b6-156003663a12-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn\" (UID: \"15ce599e-f880-46d7-91b6-156003663a12\") " pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:22:23.340753 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.340608    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5q6w\" (UniqueName: \"kubernetes.io/projected/15ce599e-f880-46d7-91b6-156003663a12-kube-api-access-q5q6w\") pod \"payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn\" (UID: \"15ce599e-f880-46d7-91b6-156003663a12\") " pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:22:23.441273 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.441243    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15ce599e-f880-46d7-91b6-156003663a12-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn\" (UID: \"15ce599e-f880-46d7-91b6-156003663a12\") " pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:22:23.441462 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.441314    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5q6w\" (UniqueName: \"kubernetes.io/projected/15ce599e-f880-46d7-91b6-156003663a12-kube-api-access-q5q6w\") pod \"payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn\" (UID: \"15ce599e-f880-46d7-91b6-156003663a12\") " pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:22:23.441957 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.441938    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15ce599e-f880-46d7-91b6-156003663a12-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn\" (UID: \"15ce599e-f880-46d7-91b6-156003663a12\") " pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:22:23.450396 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.450372    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5q6w\" (UniqueName: \"kubernetes.io/projected/15ce599e-f880-46d7-91b6-156003663a12-kube-api-access-q5q6w\") pod \"payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn\" (UID: \"15ce599e-f880-46d7-91b6-156003663a12\") " pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:22:23.582839 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.582748    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:22:23.861379 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:23.861343    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"]
Sep 08 14:22:23.863451 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:22:23.863415    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ce599e_f880_46d7_91b6_156003663a12.slice/crio-4063fafc1839823dedea254fc2416b759da070b3d91a77ad09e0c6bcde38a581 WatchSource:0}: Error finding container 4063fafc1839823dedea254fc2416b759da070b3d91a77ad09e0c6bcde38a581: Status 404 returned error can't find the container with id 4063fafc1839823dedea254fc2416b759da070b3d91a77ad09e0c6bcde38a581
Sep 08 14:22:24.441475 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:24.441439    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn" event={"ID":"15ce599e-f880-46d7-91b6-156003663a12","Type":"ContainerStarted","Data":"92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030"}
Sep 08 14:22:24.442025 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:24.441498    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn" event={"ID":"15ce599e-f880-46d7-91b6-156003663a12","Type":"ContainerStarted","Data":"4063fafc1839823dedea254fc2416b759da070b3d91a77ad09e0c6bcde38a581"}
Sep 08 14:22:24.442025 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:24.441527    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:22:24.459481 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:24.459405    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn" podStartSLOduration=1.459388112 podStartE2EDuration="1.459388112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:22:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:22:24.458068107 +0000 UTC m=+1434.953038792" watchObservedRunningTime="2026-09-08 14:22:24.459388112 +0000 UTC m=+1434.954358796"
Sep 08 14:22:30.270763 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:30.270728    2564 scope.go:117] "RemoveContainer" containerID="bbbff0c0f6559bbb16ce431f260ae073c40f5cf68fb34db6b845e3b36ad02ea4"
Sep 08 14:22:35.449171 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:35.449121    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:22:44.545604 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:44.545567    2564 generic.go:356] "Generic (PLEG): container finished" podID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerID="b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079" exitCode=0
Sep 08 14:22:44.545604 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:44.545606    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" event={"ID":"d8d17bf9-6491-4504-8cfd-e3f9eba2f417","Type":"ContainerDied","Data":"b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079"}
Sep 08 14:22:44.546131 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:44.546036    2564 scope.go:117] "RemoveContainer" containerID="b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079"
Sep 08 14:22:44.547216 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:44.547181    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:22:44.547336 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:44.547251    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:22:44.547336 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:44.547286    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 14:22:45.560880 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:45.560839    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" event={"ID":"d8d17bf9-6491-4504-8cfd-e3f9eba2f417","Type":"ContainerStarted","Data":"405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548"}
Sep 08 14:22:46.404055 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:46.404014    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:46.405326 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:46.405294    2564 patch_prober.go:28] interesting pod/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 08 14:22:46.405454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:46.405371    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 08 14:22:47.404389 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:47.404353    2564 patch_prober.go:28] interesting pod/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 08 14:22:47.404766 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:47.404416    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 08 14:22:48.403760 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:48.403730    2564 patch_prober.go:28] interesting pod/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 08 14:22:48.403959 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:48.403799    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 08 14:22:49.404241 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:49.404207    2564 patch_prober.go:28] interesting pod/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 08 14:22:49.404641 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:49.404265    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 08 14:22:49.550761 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:49.550726    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"]
Sep 08 14:22:49.554944 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:49.554916    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"
Sep 08 14:22:49.557612 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:49.557587    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:22:49.557612 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:49.557601    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:22:49.564355 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:49.564331    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"]
Sep 08 14:22:49.600321 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:49.600287    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07b2f0f-5169-4cf4-ab8e-cf218833a056-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd\" (UID: \"a07b2f0f-5169-4cf4-ab8e-cf218833a056\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"
Sep 08 14:22:49.701924 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:49.701839    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07b2f0f-5169-4cf4-ab8e-cf218833a056-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd\" (UID: \"a07b2f0f-5169-4cf4-ab8e-cf218833a056\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"
Sep 08 14:22:49.702555 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:49.702530    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07b2f0f-5169-4cf4-ab8e-cf218833a056-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd\" (UID: \"a07b2f0f-5169-4cf4-ab8e-cf218833a056\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"
Sep 08 14:22:49.867247 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:49.867212    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"
Sep 08 14:22:50.141923 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:50.141888    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"]
Sep 08 14:22:50.404386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:50.404289    2564 patch_prober.go:28] interesting pod/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 08 14:22:50.404386 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:50.404358    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 08 14:22:50.416053 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:50.416028    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:50.603378 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:50.603338    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd" event={"ID":"a07b2f0f-5169-4cf4-ab8e-cf218833a056","Type":"ContainerStarted","Data":"4cf7e4e84b27a521458f4993bf273fdc639273db1a1377d959e3200376000141"}
Sep 08 14:22:50.603587 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:50.603416    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd" event={"ID":"a07b2f0f-5169-4cf4-ab8e-cf218833a056","Type":"ContainerStarted","Data":"39527c4784c85f803d44025cda40de920660d37a69e61d10cbc51cf99258857c"}
Sep 08 14:22:50.630890 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:50.624353    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd" podStartSLOduration=1.62433293 podStartE2EDuration="1.62433293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:22:50.621458396 +0000 UTC m=+1461.116429082" watchObservedRunningTime="2026-09-08 14:22:50.62433293 +0000 UTC m=+1461.119303615"
Sep 08 14:22:51.408015 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:51.407934    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:51.408794 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:51.408774    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:22:51.610639 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:51.610605    2564 generic.go:356] "Generic (PLEG): container finished" podID="a07b2f0f-5169-4cf4-ab8e-cf218833a056" containerID="4cf7e4e84b27a521458f4993bf273fdc639273db1a1377d959e3200376000141" exitCode=0
Sep 08 14:22:51.610830 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:51.610700    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd" event={"ID":"a07b2f0f-5169-4cf4-ab8e-cf218833a056","Type":"ContainerDied","Data":"4cf7e4e84b27a521458f4993bf273fdc639273db1a1377d959e3200376000141"}
Sep 08 14:22:52.751525 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:52.751501    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"
Sep 08 14:22:52.828802 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:52.828772    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07b2f0f-5169-4cf4-ab8e-cf218833a056-maas-api-service-ca\") pod \"a07b2f0f-5169-4cf4-ab8e-cf218833a056\" (UID: \"a07b2f0f-5169-4cf4-ab8e-cf218833a056\") "
Sep 08 14:22:52.829163 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:52.829140    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a07b2f0f-5169-4cf4-ab8e-cf218833a056-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a07b2f0f-5169-4cf4-ab8e-cf218833a056" (UID: "a07b2f0f-5169-4cf4-ab8e-cf218833a056"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:22:52.930213 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:52.930132    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07b2f0f-5169-4cf4-ab8e-cf218833a056-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:22:53.622474 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:53.622442    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd" event={"ID":"a07b2f0f-5169-4cf4-ab8e-cf218833a056","Type":"ContainerDied","Data":"39527c4784c85f803d44025cda40de920660d37a69e61d10cbc51cf99258857c"}
Sep 08 14:22:53.622474 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:53.622465    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"
Sep 08 14:22:53.622695 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:22:53.622492    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39527c4784c85f803d44025cda40de920660d37a69e61d10cbc51cf99258857c"
Sep 08 14:23:05.457668 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:05.457638    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"]
Sep 08 14:23:05.470711 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:05.470682    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a8b757-h7dnd"]
Sep 08 14:23:05.959414 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:05.959382    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a07b2f0f-5169-4cf4-ab8e-cf218833a056" path="/var/lib/kubelet/pods/a07b2f0f-5169-4cf4-ab8e-cf218833a056/volumes"
Sep 08 14:23:12.496300 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:12.496265    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"]
Sep 08 14:23:12.496746 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:12.496515    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy" containerID="cri-o://7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5" gracePeriod=30
Sep 08 14:23:12.848123 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:12.848037    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"]
Sep 08 14:23:12.848451 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:12.848438    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a07b2f0f-5169-4cf4-ab8e-cf218833a056" containerName="revoke-keys"
Sep 08 14:23:12.848530 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:12.848454    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a07b2f0f-5169-4cf4-ab8e-cf218833a056" containerName="revoke-keys"
Sep 08 14:23:12.848584 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:12.848534    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a07b2f0f-5169-4cf4-ab8e-cf218833a056" containerName="revoke-keys"
Sep 08 14:23:12.850927 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:12.850907    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"
Sep 08 14:23:12.853460 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:12.853431    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:23:12.853582 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:12.853496    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:23:12.858822 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:12.858795    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"]
Sep 08 14:23:12.933294 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:12.933254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b66ba9-bf2d-460c-847b-59ec0be8acb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7\" (UID: \"48b66ba9-bf2d-460c-847b-59ec0be8acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"
Sep 08 14:23:13.034962 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:13.034927    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b66ba9-bf2d-460c-847b-59ec0be8acb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7\" (UID: \"48b66ba9-bf2d-460c-847b-59ec0be8acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"
Sep 08 14:23:13.035607 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:13.035579    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b66ba9-bf2d-460c-847b-59ec0be8acb8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7\" (UID: \"48b66ba9-bf2d-460c-847b-59ec0be8acb8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"
Sep 08 14:23:13.163379 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:13.163289    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"
Sep 08 14:23:13.438986 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:13.438821    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"]
Sep 08 14:23:13.441737 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:23:13.441712    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b66ba9_bf2d_460c_847b_59ec0be8acb8.slice/crio-c3aaed2a6090f6746ba4e5d522a9136f4d7d0b98273ce1c40fdad8548a310b67 WatchSource:0}: Error finding container c3aaed2a6090f6746ba4e5d522a9136f4d7d0b98273ce1c40fdad8548a310b67: Status 404 returned error can't find the container with id c3aaed2a6090f6746ba4e5d522a9136f4d7d0b98273ce1c40fdad8548a310b67
Sep 08 14:23:13.726051 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:13.726013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7" event={"ID":"48b66ba9-bf2d-460c-847b-59ec0be8acb8","Type":"ContainerStarted","Data":"c3aaed2a6090f6746ba4e5d522a9136f4d7d0b98273ce1c40fdad8548a310b67"}
Sep 08 14:23:14.732938 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:14.732890    2564 generic.go:356] "Generic (PLEG): container finished" podID="48b66ba9-bf2d-460c-847b-59ec0be8acb8" containerID="972962a11cb27fc9620bf7165f01089d43bf53d7ead919abdb221d70225176f7" exitCode=0
Sep 08 14:23:14.733333 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:14.733036    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7" event={"ID":"48b66ba9-bf2d-460c-847b-59ec0be8acb8","Type":"ContainerDied","Data":"972962a11cb27fc9620bf7165f01089d43bf53d7ead919abdb221d70225176f7"}
Sep 08 14:23:15.877187 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:15.877162    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"
Sep 08 14:23:15.964747 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:15.964720    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b66ba9-bf2d-460c-847b-59ec0be8acb8-maas-api-service-ca\") pod \"48b66ba9-bf2d-460c-847b-59ec0be8acb8\" (UID: \"48b66ba9-bf2d-460c-847b-59ec0be8acb8\") "
Sep 08 14:23:15.965143 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:15.965120    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48b66ba9-bf2d-460c-847b-59ec0be8acb8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "48b66ba9-bf2d-460c-847b-59ec0be8acb8" (UID: "48b66ba9-bf2d-460c-847b-59ec0be8acb8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:23:15.965240 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:15.965227    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b66ba9-bf2d-460c-847b-59ec0be8acb8-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:16.745488 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:16.745456    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"
Sep 08 14:23:16.745488 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:16.745479    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7" event={"ID":"48b66ba9-bf2d-460c-847b-59ec0be8acb8","Type":"ContainerDied","Data":"c3aaed2a6090f6746ba4e5d522a9136f4d7d0b98273ce1c40fdad8548a310b67"}
Sep 08 14:23:16.745697 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:16.745520    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3aaed2a6090f6746ba4e5d522a9136f4d7d0b98273ce1c40fdad8548a310b67"
Sep 08 14:23:17.745306 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.745283    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:23:17.751605 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.751577    2564 generic.go:356] "Generic (PLEG): container finished" podID="4e8781c7-053e-4317-b816-fc23d9c000da" containerID="7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5" exitCode=0
Sep 08 14:23:17.751703 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.751618    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"
Sep 08 14:23:17.751703 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.751615    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" event={"ID":"4e8781c7-053e-4317-b816-fc23d9c000da","Type":"ContainerDied","Data":"7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5"}
Sep 08 14:23:17.751783 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.751730    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd" event={"ID":"4e8781c7-053e-4317-b816-fc23d9c000da","Type":"ContainerDied","Data":"649679c8af69e89bd67e396e0a3c9fa45131bfc3dc7841af052ac3ae6ca0a33a"}
Sep 08 14:23:17.751783 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.751753    2564 scope.go:117] "RemoveContainer" containerID="7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5"
Sep 08 14:23:17.761077 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.761058    2564 scope.go:117] "RemoveContainer" containerID="5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5"
Sep 08 14:23:17.771663 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.771642    2564 scope.go:117] "RemoveContainer" containerID="7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5"
Sep 08 14:23:17.771996 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:23:17.771976    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5\": container with ID starting with 7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5 not found: ID does not exist" containerID="7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5"
Sep 08 14:23:17.772100 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.772007    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5"} err="failed to get container status \"7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5\": rpc error: code = NotFound desc = could not find container \"7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5\": container with ID starting with 7e00ff83fc8d9d9962b3c97b64b98a57830d009015a42851fe189a8512bb38d5 not found: ID does not exist"
Sep 08 14:23:17.772100 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.772032    2564 scope.go:117] "RemoveContainer" containerID="5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5"
Sep 08 14:23:17.772347 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:23:17.772330    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5\": container with ID starting with 5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5 not found: ID does not exist" containerID="5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5"
Sep 08 14:23:17.772393 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.772365    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5"} err="failed to get container status \"5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5\": rpc error: code = NotFound desc = could not find container \"5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5\": container with ID starting with 5c0e9f16a2ebf46caca7086460722fcbd4cee80e884af5153db0b2005a4210d5 not found: ID does not exist"
Sep 08 14:23:17.884180 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884136    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-certs\") pod \"4e8781c7-053e-4317-b816-fc23d9c000da\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") "
Sep 08 14:23:17.884180 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884195    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-data\") pod \"4e8781c7-053e-4317-b816-fc23d9c000da\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") "
Sep 08 14:23:17.884454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884222    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e8781c7-053e-4317-b816-fc23d9c000da-istiod-ca-cert\") pod \"4e8781c7-053e-4317-b816-fc23d9c000da\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") "
Sep 08 14:23:17.884454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884250    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-istio-token\") pod \"4e8781c7-053e-4317-b816-fc23d9c000da\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") "
Sep 08 14:23:17.884454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884297    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-envoy\") pod \"4e8781c7-053e-4317-b816-fc23d9c000da\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") "
Sep 08 14:23:17.884454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884330    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e8781c7-053e-4317-b816-fc23d9c000da-istio-podinfo\") pod \"4e8781c7-053e-4317-b816-fc23d9c000da\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") "
Sep 08 14:23:17.884454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884374    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8rhc\" (UniqueName: \"kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-kube-api-access-h8rhc\") pod \"4e8781c7-053e-4317-b816-fc23d9c000da\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") "
Sep 08 14:23:17.884454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884406    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-socket\") pod \"4e8781c7-053e-4317-b816-fc23d9c000da\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") "
Sep 08 14:23:17.884772 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884461    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-credential-socket\") pod \"4e8781c7-053e-4317-b816-fc23d9c000da\" (UID: \"4e8781c7-053e-4317-b816-fc23d9c000da\") "
Sep 08 14:23:17.884772 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884488    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e8781c7-053e-4317-b816-fc23d9c000da" (UID: "4e8781c7-053e-4317-b816-fc23d9c000da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:23:17.884772 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884722    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e8781c7-053e-4317-b816-fc23d9c000da" (UID: "4e8781c7-053e-4317-b816-fc23d9c000da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:23:17.884772 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884754    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e8781c7-053e-4317-b816-fc23d9c000da-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e8781c7-053e-4317-b816-fc23d9c000da" (UID: "4e8781c7-053e-4317-b816-fc23d9c000da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:23:17.885048 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.884802    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e8781c7-053e-4317-b816-fc23d9c000da" (UID: "4e8781c7-053e-4317-b816-fc23d9c000da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:23:17.886748 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.885626    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:17.886748 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.885651    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e8781c7-053e-4317-b816-fc23d9c000da" (UID: "4e8781c7-053e-4317-b816-fc23d9c000da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:23:17.886748 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.885661    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e8781c7-053e-4317-b816-fc23d9c000da-istiod-ca-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:17.886748 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.885701    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-workload-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:17.893776 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.892743    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e8781c7-053e-4317-b816-fc23d9c000da" (UID: "4e8781c7-053e-4317-b816-fc23d9c000da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:23:17.893776 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.892905    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-kube-api-access-h8rhc" (OuterVolumeSpecName: "kube-api-access-h8rhc") pod "4e8781c7-053e-4317-b816-fc23d9c000da" (UID: "4e8781c7-053e-4317-b816-fc23d9c000da"). InnerVolumeSpecName "kube-api-access-h8rhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:23:17.893776 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.892919    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e8781c7-053e-4317-b816-fc23d9c000da-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e8781c7-053e-4317-b816-fc23d9c000da" (UID: "4e8781c7-053e-4317-b816-fc23d9c000da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:23:17.894101 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.893925    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e8781c7-053e-4317-b816-fc23d9c000da" (UID: "4e8781c7-053e-4317-b816-fc23d9c000da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:23:17.986717 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.986690    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-envoy\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:17.986717 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.986714    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e8781c7-053e-4317-b816-fc23d9c000da-istio-podinfo\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:17.986889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.986723    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8rhc\" (UniqueName: \"kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-kube-api-access-h8rhc\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:17.986889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.986733    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-credential-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:17.986889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.986741    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e8781c7-053e-4317-b816-fc23d9c000da-istio-data\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:17.986889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:17.986749    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e8781c7-053e-4317-b816-fc23d9c000da-istio-token\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:18.070220 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:18.070123    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"]
Sep 08 14:23:18.073783 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:18.073751    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a8b757-openshift-default-5ffc86899b-5v4zd"]
Sep 08 14:23:19.960832 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:19.960801    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" path="/var/lib/kubelet/pods/4e8781c7-053e-4317-b816-fc23d9c000da/volumes"
Sep 08 14:23:28.413184 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.413149    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"]
Sep 08 14:23:28.413595 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.413434    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn" podUID="15ce599e-f880-46d7-91b6-156003663a12" containerName="payload-processing" containerID="cri-o://92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030" gracePeriod=30
Sep 08 14:23:28.654642 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.654571    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"]
Sep 08 14:23:28.657686 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.657657    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-2310c4-pjkd7"]
Sep 08 14:23:28.683165 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.683141    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:23:28.686336 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.686317    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5q6w\" (UniqueName: \"kubernetes.io/projected/15ce599e-f880-46d7-91b6-156003663a12-kube-api-access-q5q6w\") pod \"15ce599e-f880-46d7-91b6-156003663a12\" (UID: \"15ce599e-f880-46d7-91b6-156003663a12\") "
Sep 08 14:23:28.686480 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.686354    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15ce599e-f880-46d7-91b6-156003663a12-plugins-config-volume\") pod \"15ce599e-f880-46d7-91b6-156003663a12\" (UID: \"15ce599e-f880-46d7-91b6-156003663a12\") "
Sep 08 14:23:28.686704 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.686683    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15ce599e-f880-46d7-91b6-156003663a12-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "15ce599e-f880-46d7-91b6-156003663a12" (UID: "15ce599e-f880-46d7-91b6-156003663a12"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:23:28.688610 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.688584    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ce599e-f880-46d7-91b6-156003663a12-kube-api-access-q5q6w" (OuterVolumeSpecName: "kube-api-access-q5q6w") pod "15ce599e-f880-46d7-91b6-156003663a12" (UID: "15ce599e-f880-46d7-91b6-156003663a12"). InnerVolumeSpecName "kube-api-access-q5q6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:23:28.787079 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.787038    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5q6w\" (UniqueName: \"kubernetes.io/projected/15ce599e-f880-46d7-91b6-156003663a12-kube-api-access-q5q6w\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:28.787079 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.787074    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15ce599e-f880-46d7-91b6-156003663a12-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:28.811840 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.811802    2564 generic.go:356] "Generic (PLEG): container finished" podID="15ce599e-f880-46d7-91b6-156003663a12" containerID="92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030" exitCode=0
Sep 08 14:23:28.812018 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.811847    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"
Sep 08 14:23:28.812018 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.811891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn" event={"ID":"15ce599e-f880-46d7-91b6-156003663a12","Type":"ContainerDied","Data":"92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030"}
Sep 08 14:23:28.812018 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.811973    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn" event={"ID":"15ce599e-f880-46d7-91b6-156003663a12","Type":"ContainerDied","Data":"4063fafc1839823dedea254fc2416b759da070b3d91a77ad09e0c6bcde38a581"}
Sep 08 14:23:28.812018 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.812000    2564 scope.go:117] "RemoveContainer" containerID="92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030"
Sep 08 14:23:28.821582 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.821565    2564 scope.go:117] "RemoveContainer" containerID="92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030"
Sep 08 14:23:28.821854 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:23:28.821835    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030\": container with ID starting with 92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030 not found: ID does not exist" containerID="92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030"
Sep 08 14:23:28.821939 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.821882    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030"} err="failed to get container status \"92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030\": rpc error: code = NotFound desc = could not find container \"92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030\": container with ID starting with 92b2bdf01dde86f96004f310b8ac04dceeeb5058eb88aa71a9b0efeeae7ce030 not found: ID does not exist"
Sep 08 14:23:28.833859 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.833833    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"]
Sep 08 14:23:28.840366 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:28.840346    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2310c4-64795d7c67-9zjsn"]
Sep 08 14:23:29.958926 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:29.958891    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15ce599e-f880-46d7-91b6-156003663a12" path="/var/lib/kubelet/pods/15ce599e-f880-46d7-91b6-156003663a12/volumes"
Sep 08 14:23:29.959287 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:29.959245    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48b66ba9-bf2d-460c-847b-59ec0be8acb8" path="/var/lib/kubelet/pods/48b66ba9-bf2d-460c-847b-59ec0be8acb8/volumes"
Sep 08 14:23:33.897103 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:33.897075    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:23:33.906586 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:33.906563    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:23:35.817466 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:35.817434    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"]
Sep 08 14:23:35.817825 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:35.817642    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy" containerID="cri-o://405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548" gracePeriod=30
Sep 08 14:23:36.410936 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:36.410903    2564 patch_prober.go:28] interesting pod/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 14:23:36.411113 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:36.410959    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 14:23:41.066178 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.066152    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:23:41.101973 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.101903    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istiod-ca-cert\") pod \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") "
Sep 08 14:23:41.101973 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.101935    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-credential-socket\") pod \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") "
Sep 08 14:23:41.102188 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.101985    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-envoy\") pod \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") "
Sep 08 14:23:41.102188 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102013    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-token\") pod \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") "
Sep 08 14:23:41.102188 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102116    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pllm8\" (UniqueName: \"kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-kube-api-access-pllm8\") pod \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") "
Sep 08 14:23:41.102188 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102178    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-data\") pod \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") "
Sep 08 14:23:41.102395 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102201    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-certs\") pod \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") "
Sep 08 14:23:41.102395 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102237    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-socket\") pod \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") "
Sep 08 14:23:41.102395 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102266    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-podinfo\") pod \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\" (UID: \"d8d17bf9-6491-4504-8cfd-e3f9eba2f417\") "
Sep 08 14:23:41.102395 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102321    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d8d17bf9-6491-4504-8cfd-e3f9eba2f417" (UID: "d8d17bf9-6491-4504-8cfd-e3f9eba2f417"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:23:41.102395 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102327    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d8d17bf9-6491-4504-8cfd-e3f9eba2f417" (UID: "d8d17bf9-6491-4504-8cfd-e3f9eba2f417"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:23:41.102659 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102574    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istiod-ca-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:41.102659 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102598    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-credential-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:41.102659 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102590    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-data" (OuterVolumeSpecName: "istio-data") pod "d8d17bf9-6491-4504-8cfd-e3f9eba2f417" (UID: "d8d17bf9-6491-4504-8cfd-e3f9eba2f417"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:23:41.102818 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102646    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d8d17bf9-6491-4504-8cfd-e3f9eba2f417" (UID: "d8d17bf9-6491-4504-8cfd-e3f9eba2f417"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:23:41.102818 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.102791    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d8d17bf9-6491-4504-8cfd-e3f9eba2f417" (UID: "d8d17bf9-6491-4504-8cfd-e3f9eba2f417"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:23:41.104679 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.104652    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d8d17bf9-6491-4504-8cfd-e3f9eba2f417" (UID: "d8d17bf9-6491-4504-8cfd-e3f9eba2f417"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:23:41.104920 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.104857    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-kube-api-access-pllm8" (OuterVolumeSpecName: "kube-api-access-pllm8") pod "d8d17bf9-6491-4504-8cfd-e3f9eba2f417" (UID: "d8d17bf9-6491-4504-8cfd-e3f9eba2f417"). InnerVolumeSpecName "kube-api-access-pllm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:23:41.105041 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.104968    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d8d17bf9-6491-4504-8cfd-e3f9eba2f417" (UID: "d8d17bf9-6491-4504-8cfd-e3f9eba2f417"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:23:41.105041 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.104988    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-token" (OuterVolumeSpecName: "istio-token") pod "d8d17bf9-6491-4504-8cfd-e3f9eba2f417" (UID: "d8d17bf9-6491-4504-8cfd-e3f9eba2f417"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:23:41.204091 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.204061    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pllm8\" (UniqueName: \"kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-kube-api-access-pllm8\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:41.204091 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.204089    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-data\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:41.204268 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.204098    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:41.204268 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.204107    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-workload-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:41.204268 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.204114    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-podinfo\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:41.204268 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.204123    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-envoy\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:41.204268 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.204131    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8d17bf9-6491-4504-8cfd-e3f9eba2f417-istio-token\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:41.883847 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.883808    2564 generic.go:356] "Generic (PLEG): container finished" podID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerID="405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548" exitCode=0
Sep 08 14:23:41.884080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.883881    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"
Sep 08 14:23:41.884080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.883899    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" event={"ID":"d8d17bf9-6491-4504-8cfd-e3f9eba2f417","Type":"ContainerDied","Data":"405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548"}
Sep 08 14:23:41.884080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.883974    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m" event={"ID":"d8d17bf9-6491-4504-8cfd-e3f9eba2f417","Type":"ContainerDied","Data":"7c2fab1a15afdec663f0268d31b7c3f249baac2a7aedf923b1caaa7e3b6a64c4"}
Sep 08 14:23:41.884080 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.884006    2564 scope.go:117] "RemoveContainer" containerID="405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548"
Sep 08 14:23:41.894679 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.894655    2564 scope.go:117] "RemoveContainer" containerID="b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079"
Sep 08 14:23:41.904502 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.904477    2564 scope.go:117] "RemoveContainer" containerID="405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548"
Sep 08 14:23:41.904922 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:23:41.904888    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548\": container with ID starting with 405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548 not found: ID does not exist" containerID="405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548"
Sep 08 14:23:41.905021 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.904931    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548"} err="failed to get container status \"405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548\": rpc error: code = NotFound desc = could not find container \"405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548\": container with ID starting with 405375bc1561f3207e85f5282b9038a30f7ba7b2954eac3191ab093048ece548 not found: ID does not exist"
Sep 08 14:23:41.905021 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.904955    2564 scope.go:117] "RemoveContainer" containerID="b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079"
Sep 08 14:23:41.905310 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:23:41.905255    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079\": container with ID starting with b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079 not found: ID does not exist" containerID="b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079"
Sep 08 14:23:41.905310 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.905281    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079"} err="failed to get container status \"b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079\": rpc error: code = NotFound desc = could not find container \"b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079\": container with ID starting with b4c8b2401b7d1709620a1bd8e0435e80a7e1d93e094211af9d4537c0314a5079 not found: ID does not exist"
Sep 08 14:23:41.909412 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.909384    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"]
Sep 08 14:23:41.912674 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.912652    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-2310c4-openshift-default-64f8dd795b-l2b6m"]
Sep 08 14:23:41.959695 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:41.959659    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" path="/var/lib/kubelet/pods/d8d17bf9-6491-4504-8cfd-e3f9eba2f417/volumes"
Sep 08 14:23:46.802938 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:46.802902    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"]
Sep 08 14:23:46.803515 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:46.803174    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2" podUID="bf2fb49b-e774-41c8-8ea4-14daa18591e8" containerName="payload-processing" containerID="cri-o://e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9" gracePeriod=30
Sep 08 14:23:46.811506 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:46.811478    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"]
Sep 08 14:23:46.811715 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:46.811688    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl" podUID="576a03f2-3518-4a1b-bdfa-858583427e84" containerName="payload-pre-processing" containerID="cri-o://b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8" gracePeriod=30
Sep 08 14:23:47.086674 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.086647    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:23:47.089561 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.089543    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:23:47.160731 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.160689    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf2fb49b-e774-41c8-8ea4-14daa18591e8-plugins-config-volume\") pod \"bf2fb49b-e774-41c8-8ea4-14daa18591e8\" (UID: \"bf2fb49b-e774-41c8-8ea4-14daa18591e8\") "
Sep 08 14:23:47.160944 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.160759    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/576a03f2-3518-4a1b-bdfa-858583427e84-plugins-config-volume\") pod \"576a03f2-3518-4a1b-bdfa-858583427e84\" (UID: \"576a03f2-3518-4a1b-bdfa-858583427e84\") "
Sep 08 14:23:47.160944 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.160827    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8xkr\" (UniqueName: \"kubernetes.io/projected/576a03f2-3518-4a1b-bdfa-858583427e84-kube-api-access-j8xkr\") pod \"576a03f2-3518-4a1b-bdfa-858583427e84\" (UID: \"576a03f2-3518-4a1b-bdfa-858583427e84\") "
Sep 08 14:23:47.160944 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.160890    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6d275\" (UniqueName: \"kubernetes.io/projected/bf2fb49b-e774-41c8-8ea4-14daa18591e8-kube-api-access-6d275\") pod \"bf2fb49b-e774-41c8-8ea4-14daa18591e8\" (UID: \"bf2fb49b-e774-41c8-8ea4-14daa18591e8\") "
Sep 08 14:23:47.161152 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.161121    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf2fb49b-e774-41c8-8ea4-14daa18591e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf2fb49b-e774-41c8-8ea4-14daa18591e8" (UID: "bf2fb49b-e774-41c8-8ea4-14daa18591e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:23:47.161198 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.161148    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/576a03f2-3518-4a1b-bdfa-858583427e84-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "576a03f2-3518-4a1b-bdfa-858583427e84" (UID: "576a03f2-3518-4a1b-bdfa-858583427e84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:23:47.161291 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.161276    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/576a03f2-3518-4a1b-bdfa-858583427e84-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:47.161329 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.161294    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf2fb49b-e774-41c8-8ea4-14daa18591e8-plugins-config-volume\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:47.163114 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.163091    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/576a03f2-3518-4a1b-bdfa-858583427e84-kube-api-access-j8xkr" (OuterVolumeSpecName: "kube-api-access-j8xkr") pod "576a03f2-3518-4a1b-bdfa-858583427e84" (UID: "576a03f2-3518-4a1b-bdfa-858583427e84"). InnerVolumeSpecName "kube-api-access-j8xkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:23:47.163114 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.163097    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf2fb49b-e774-41c8-8ea4-14daa18591e8-kube-api-access-6d275" (OuterVolumeSpecName: "kube-api-access-6d275") pod "bf2fb49b-e774-41c8-8ea4-14daa18591e8" (UID: "bf2fb49b-e774-41c8-8ea4-14daa18591e8"). InnerVolumeSpecName "kube-api-access-6d275". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:23:47.262209 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.262173    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8xkr\" (UniqueName: \"kubernetes.io/projected/576a03f2-3518-4a1b-bdfa-858583427e84-kube-api-access-j8xkr\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:47.262209 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.262200    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d275\" (UniqueName: \"kubernetes.io/projected/bf2fb49b-e774-41c8-8ea4-14daa18591e8-kube-api-access-6d275\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:47.917647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.917617    2564 generic.go:356] "Generic (PLEG): container finished" podID="bf2fb49b-e774-41c8-8ea4-14daa18591e8" containerID="e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9" exitCode=0
Sep 08 14:23:47.918142 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.917674    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"
Sep 08 14:23:47.918142 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.917699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2" event={"ID":"bf2fb49b-e774-41c8-8ea4-14daa18591e8","Type":"ContainerDied","Data":"e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9"}
Sep 08 14:23:47.918142 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.917764    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2" event={"ID":"bf2fb49b-e774-41c8-8ea4-14daa18591e8","Type":"ContainerDied","Data":"95ec62914e8b662c6bad4b75b94a181314f288f9091bd4eb7f640a74ee390b36"}
Sep 08 14:23:47.918142 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.917794    2564 scope.go:117] "RemoveContainer" containerID="e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9"
Sep 08 14:23:47.919380 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.919357    2564 generic.go:356] "Generic (PLEG): container finished" podID="576a03f2-3518-4a1b-bdfa-858583427e84" containerID="b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8" exitCode=0
Sep 08 14:23:47.919539 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.919404    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"
Sep 08 14:23:47.919539 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.919413    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl" event={"ID":"576a03f2-3518-4a1b-bdfa-858583427e84","Type":"ContainerDied","Data":"b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8"}
Sep 08 14:23:47.919539 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.919448    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl" event={"ID":"576a03f2-3518-4a1b-bdfa-858583427e84","Type":"ContainerDied","Data":"2bd6c2866fcc81dc7674ddec9e5400a392e097f113dbf07151b1050c34ee5854"}
Sep 08 14:23:47.932711 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.932693    2564 scope.go:117] "RemoveContainer" containerID="e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9"
Sep 08 14:23:47.933389 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:23:47.933362    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9\": container with ID starting with e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9 not found: ID does not exist" containerID="e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9"
Sep 08 14:23:47.933502 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.933401    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9"} err="failed to get container status \"e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9\": rpc error: code = NotFound desc = could not find container \"e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9\": container with ID starting with e266365bf99cfb4c1fb8f5d80ce7f2d10ef7792f7a44693c5499043c9e34a7e9 not found: ID does not exist"
Sep 08 14:23:47.933502 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.933429    2564 scope.go:117] "RemoveContainer" containerID="b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8"
Sep 08 14:23:47.941957 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.941939    2564 scope.go:117] "RemoveContainer" containerID="b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8"
Sep 08 14:23:47.942222 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:23:47.942205    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8\": container with ID starting with b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8 not found: ID does not exist" containerID="b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8"
Sep 08 14:23:47.942274 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.942228    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8"} err="failed to get container status \"b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8\": rpc error: code = NotFound desc = could not find container \"b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8\": container with ID starting with b5b75331fb92b322c809b68e46d8435c8813c584dbc265d90e9b1863086619f8 not found: ID does not exist"
Sep 08 14:23:47.949760 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.949739    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"]
Sep 08 14:23:47.959207 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.959187    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2bcb20-6dc6b97899-zw4g2"]
Sep 08 14:23:47.965311 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.965293    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"]
Sep 08 14:23:47.974747 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:47.974720    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-2bcb20-85f5665d78-dltwl"]
Sep 08 14:23:49.959621 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:49.959589    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="576a03f2-3518-4a1b-bdfa-858583427e84" path="/var/lib/kubelet/pods/576a03f2-3518-4a1b-bdfa-858583427e84/volumes"
Sep 08 14:23:49.960058 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:49.959961    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf2fb49b-e774-41c8-8ea4-14daa18591e8" path="/var/lib/kubelet/pods/bf2fb49b-e774-41c8-8ea4-14daa18591e8/volumes"
Sep 08 14:23:54.348627 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.348524    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"]
Sep 08 14:23:54.349209 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349187    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy"
Sep 08 14:23:54.349303 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349212    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy"
Sep 08 14:23:54.349303 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349231    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf2fb49b-e774-41c8-8ea4-14daa18591e8" containerName="payload-processing"
Sep 08 14:23:54.349303 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349240    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf2fb49b-e774-41c8-8ea4-14daa18591e8" containerName="payload-processing"
Sep 08 14:23:54.349303 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349255    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy"
Sep 08 14:23:54.349303 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349266    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy"
Sep 08 14:23:54.349303 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349281    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48b66ba9-bf2d-460c-847b-59ec0be8acb8" containerName="revoke-keys"
Sep 08 14:23:54.349303 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349290    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="48b66ba9-bf2d-460c-847b-59ec0be8acb8" containerName="revoke-keys"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349308    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="576a03f2-3518-4a1b-bdfa-858583427e84" containerName="payload-pre-processing"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349319    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="576a03f2-3518-4a1b-bdfa-858583427e84" containerName="payload-pre-processing"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349332    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15ce599e-f880-46d7-91b6-156003663a12" containerName="payload-processing"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349340    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="15ce599e-f880-46d7-91b6-156003663a12" containerName="payload-processing"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349356    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349365    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349382    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349391    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349490    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="48b66ba9-bf2d-460c-847b-59ec0be8acb8" containerName="revoke-keys"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349501    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf2fb49b-e774-41c8-8ea4-14daa18591e8" containerName="payload-processing"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349513    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="15ce599e-f880-46d7-91b6-156003663a12" containerName="payload-processing"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349522    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349530    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349540    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8d17bf9-6491-4504-8cfd-e3f9eba2f417" containerName="istio-proxy"
Sep 08 14:23:54.349660 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.349550    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="576a03f2-3518-4a1b-bdfa-858583427e84" containerName="payload-pre-processing"
Sep 08 14:23:54.354148 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.354126    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"
Sep 08 14:23:54.357090 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.357072    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 14:23:54.357409 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.357395    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:23:54.362818 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.362788    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"]
Sep 08 14:23:54.433119 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.433078    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8a373f0-9895-4b6c-b37b-e423ba7bf809-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5\" (UID: \"e8a373f0-9895-4b6c-b37b-e423ba7bf809\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"
Sep 08 14:23:54.534699 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.534667    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8a373f0-9895-4b6c-b37b-e423ba7bf809-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5\" (UID: \"e8a373f0-9895-4b6c-b37b-e423ba7bf809\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"
Sep 08 14:23:54.535306 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.535285    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8a373f0-9895-4b6c-b37b-e423ba7bf809-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5\" (UID: \"e8a373f0-9895-4b6c-b37b-e423ba7bf809\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"
Sep 08 14:23:54.666061 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.665962    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"
Sep 08 14:23:54.939167 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.939145    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"]
Sep 08 14:23:54.941677 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:23:54.941652    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8a373f0_9895_4b6c_b37b_e423ba7bf809.slice/crio-63658ce309803a066a67e2de7ecf162a5f3a2c4f7e467562333205d0a6192f6e WatchSource:0}: Error finding container 63658ce309803a066a67e2de7ecf162a5f3a2c4f7e467562333205d0a6192f6e: Status 404 returned error can't find the container with id 63658ce309803a066a67e2de7ecf162a5f3a2c4f7e467562333205d0a6192f6e
Sep 08 14:23:54.961369 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:54.961343    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5" event={"ID":"e8a373f0-9895-4b6c-b37b-e423ba7bf809","Type":"ContainerStarted","Data":"63658ce309803a066a67e2de7ecf162a5f3a2c4f7e467562333205d0a6192f6e"}
Sep 08 14:23:55.967527 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:55.967495    2564 generic.go:356] "Generic (PLEG): container finished" podID="e8a373f0-9895-4b6c-b37b-e423ba7bf809" containerID="0463bfb015950497b81584448964c6f008f8d50c1ddcc76634b4ee95074a180a" exitCode=0
Sep 08 14:23:55.967929 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:55.967572    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5" event={"ID":"e8a373f0-9895-4b6c-b37b-e423ba7bf809","Type":"ContainerDied","Data":"0463bfb015950497b81584448964c6f008f8d50c1ddcc76634b4ee95074a180a"}
Sep 08 14:23:57.101644 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:57.101611    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"
Sep 08 14:23:57.264819 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:57.264737    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8a373f0-9895-4b6c-b37b-e423ba7bf809-maas-api-service-ca\") pod \"e8a373f0-9895-4b6c-b37b-e423ba7bf809\" (UID: \"e8a373f0-9895-4b6c-b37b-e423ba7bf809\") "
Sep 08 14:23:57.265182 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:57.265163    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a373f0-9895-4b6c-b37b-e423ba7bf809-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8a373f0-9895-4b6c-b37b-e423ba7bf809" (UID: "e8a373f0-9895-4b6c-b37b-e423ba7bf809"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:23:57.366300 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:57.366265    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8a373f0-9895-4b6c-b37b-e423ba7bf809-maas-api-service-ca\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:23:57.978987 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:57.978955    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"
Sep 08 14:23:57.979139 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:57.978992    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5" event={"ID":"e8a373f0-9895-4b6c-b37b-e423ba7bf809","Type":"ContainerDied","Data":"63658ce309803a066a67e2de7ecf162a5f3a2c4f7e467562333205d0a6192f6e"}
Sep 08 14:23:57.979139 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:23:57.979045    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63658ce309803a066a67e2de7ecf162a5f3a2c4f7e467562333205d0a6192f6e"
Sep 08 14:24:05.073227 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:05.073187    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"]
Sep 08 14:24:05.076397 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:05.076366    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e2a04d-h6dm5"]
Sep 08 14:24:05.959018 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:05.958985    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8a373f0-9895-4b6c-b37b-e423ba7bf809" path="/var/lib/kubelet/pods/e8a373f0-9895-4b6c-b37b-e423ba7bf809/volumes"
Sep 08 14:24:11.789874 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:11.789821    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"]
Sep 08 14:24:11.790288 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:11.790150    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy" containerID="cri-o://b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e" gracePeriod=30
Sep 08 14:24:14.401397 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:14.401357    2564 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 14:24:14.401792 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:14.401437    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 14:24:14.597292 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:14.597260    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-l49z4"]
Sep 08 14:24:14.597489 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:14.597453    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6d9494c8c4-l49z4" podUID="f2f2f32a-8fe4-4408-8b0a-0c100b1217fc" containerName="manager" containerID="cri-o://56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715" gracePeriod=30
Sep 08 14:24:14.843856 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:14.843834    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:24:14.921910 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:14.921858    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-metrics-certs\") pod \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") "
Sep 08 14:24:14.922098 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:14.921955    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-webhook-cert\") pod \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") "
Sep 08 14:24:14.922098 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:14.921987    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsxcd\" (UniqueName: \"kubernetes.io/projected/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-kube-api-access-jsxcd\") pod \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\" (UID: \"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc\") "
Sep 08 14:24:14.924093 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:14.924055    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f2f2f32a-8fe4-4408-8b0a-0c100b1217fc" (UID: "f2f2f32a-8fe4-4408-8b0a-0c100b1217fc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:24:14.924222 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:14.924172    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-kube-api-access-jsxcd" (OuterVolumeSpecName: "kube-api-access-jsxcd") pod "f2f2f32a-8fe4-4408-8b0a-0c100b1217fc" (UID: "f2f2f32a-8fe4-4408-8b0a-0c100b1217fc"). InnerVolumeSpecName "kube-api-access-jsxcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:24:14.924222 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:14.924200    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f2f2f32a-8fe4-4408-8b0a-0c100b1217fc" (UID: "f2f2f32a-8fe4-4408-8b0a-0c100b1217fc"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:24:15.023079 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.022977    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-metrics-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:15.023079 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.023016    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-webhook-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:15.023079 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.023029    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsxcd\" (UniqueName: \"kubernetes.io/projected/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc-kube-api-access-jsxcd\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:15.069877 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.069825    2564 generic.go:356] "Generic (PLEG): container finished" podID="f2f2f32a-8fe4-4408-8b0a-0c100b1217fc" containerID="56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715" exitCode=0
Sep 08 14:24:15.070051 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.069895    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9494c8c4-l49z4"
Sep 08 14:24:15.070051 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.069928    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9494c8c4-l49z4" event={"ID":"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc","Type":"ContainerDied","Data":"56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715"}
Sep 08 14:24:15.070051 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.070005    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9494c8c4-l49z4" event={"ID":"f2f2f32a-8fe4-4408-8b0a-0c100b1217fc","Type":"ContainerDied","Data":"1261b8876478638facff69d95942c918fcb347558b0eef40d99927b975f66dbf"}
Sep 08 14:24:15.070051 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.070042    2564 scope.go:117] "RemoveContainer" containerID="56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715"
Sep 08 14:24:15.079833 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.079818    2564 scope.go:117] "RemoveContainer" containerID="56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715"
Sep 08 14:24:15.080118 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:24:15.080099    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715\": container with ID starting with 56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715 not found: ID does not exist" containerID="56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715"
Sep 08 14:24:15.080174 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.080126    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715"} err="failed to get container status \"56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715\": rpc error: code = NotFound desc = could not find container \"56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715\": container with ID starting with 56e4a13024385c6f9b98fcc4ab55679a0080e07fafcb198c196254f453162715 not found: ID does not exist"
Sep 08 14:24:15.092030 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.091997    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-l49z4"]
Sep 08 14:24:15.095903 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.095881    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-l49z4"]
Sep 08 14:24:15.960613 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:15.960574    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2f2f32a-8fe4-4408-8b0a-0c100b1217fc" path="/var/lib/kubelet/pods/f2f2f32a-8fe4-4408-8b0a-0c100b1217fc/volumes"
Sep 08 14:24:17.041189 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.041167    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:24:17.085012 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.084804    2564 generic.go:356] "Generic (PLEG): container finished" podID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerID="b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e" exitCode=0
Sep 08 14:24:17.085012 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.084852    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"
Sep 08 14:24:17.085012 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.084911    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" event={"ID":"0ee5f985-7473-447b-a1f3-d886935ae0bc","Type":"ContainerDied","Data":"b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e"}
Sep 08 14:24:17.085012 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.084977    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw" event={"ID":"0ee5f985-7473-447b-a1f3-d886935ae0bc","Type":"ContainerDied","Data":"40e167a8817ef71dab54847797a0a514ed4cd56fd3f4647ad03e0321d38f5b73"}
Sep 08 14:24:17.085012 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.085002    2564 scope.go:117] "RemoveContainer" containerID="b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e"
Sep 08 14:24:17.093965 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.093928    2564 scope.go:117] "RemoveContainer" containerID="8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c"
Sep 08 14:24:17.106486 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.106451    2564 scope.go:117] "RemoveContainer" containerID="b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e"
Sep 08 14:24:17.106968 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:24:17.106942    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e\": container with ID starting with b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e not found: ID does not exist" containerID="b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e"
Sep 08 14:24:17.107092 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.106980    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e"} err="failed to get container status \"b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e\": rpc error: code = NotFound desc = could not find container \"b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e\": container with ID starting with b291c929e1f2ec3cdbf89aeeda52edb2bc5df74ce71e205e096dc8f48b4a335e not found: ID does not exist"
Sep 08 14:24:17.107092 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.107008    2564 scope.go:117] "RemoveContainer" containerID="8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c"
Sep 08 14:24:17.107414 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:24:17.107394    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c\": container with ID starting with 8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c not found: ID does not exist" containerID="8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c"
Sep 08 14:24:17.107486 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.107424    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c"} err="failed to get container status \"8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c\": rpc error: code = NotFound desc = could not find container \"8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c\": container with ID starting with 8e93c284738cc56cda8b42edc386ae43c65eaaa7dbe113fa909826c33cd31e0c not found: ID does not exist"
Sep 08 14:24:17.144507 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.144479    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-token\") pod \"0ee5f985-7473-447b-a1f3-d886935ae0bc\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") "
Sep 08 14:24:17.144647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.144525    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee5f985-7473-447b-a1f3-d886935ae0bc-istiod-ca-cert\") pod \"0ee5f985-7473-447b-a1f3-d886935ae0bc\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") "
Sep 08 14:24:17.144647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.144564    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-data\") pod \"0ee5f985-7473-447b-a1f3-d886935ae0bc\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") "
Sep 08 14:24:17.144647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.144584    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-credential-socket\") pod \"0ee5f985-7473-447b-a1f3-d886935ae0bc\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") "
Sep 08 14:24:17.144647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.144620    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-envoy\") pod \"0ee5f985-7473-447b-a1f3-d886935ae0bc\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") "
Sep 08 14:24:17.144647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.144638    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-podinfo\") pod \"0ee5f985-7473-447b-a1f3-d886935ae0bc\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") "
Sep 08 14:24:17.144923 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.144707    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-certs\") pod \"0ee5f985-7473-447b-a1f3-d886935ae0bc\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") "
Sep 08 14:24:17.144923 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.144750    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-socket\") pod \"0ee5f985-7473-447b-a1f3-d886935ae0bc\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") "
Sep 08 14:24:17.144923 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.144776    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6zzw\" (UniqueName: \"kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-kube-api-access-h6zzw\") pod \"0ee5f985-7473-447b-a1f3-d886935ae0bc\" (UID: \"0ee5f985-7473-447b-a1f3-d886935ae0bc\") "
Sep 08 14:24:17.145089 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.144963    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0ee5f985-7473-447b-a1f3-d886935ae0bc" (UID: "0ee5f985-7473-447b-a1f3-d886935ae0bc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:24:17.145089 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.144977    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-data" (OuterVolumeSpecName: "istio-data") pod "0ee5f985-7473-447b-a1f3-d886935ae0bc" (UID: "0ee5f985-7473-447b-a1f3-d886935ae0bc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:24:17.145089 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.145053    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ee5f985-7473-447b-a1f3-d886935ae0bc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0ee5f985-7473-447b-a1f3-d886935ae0bc" (UID: "0ee5f985-7473-447b-a1f3-d886935ae0bc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:24:17.145247 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.145189    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-data\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:17.145247 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.145211    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-credential-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:17.145247 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.145230    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ee5f985-7473-447b-a1f3-d886935ae0bc-istiod-ca-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:17.145378 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.145209    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0ee5f985-7473-447b-a1f3-d886935ae0bc" (UID: "0ee5f985-7473-447b-a1f3-d886935ae0bc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:24:17.145378 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.145313    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0ee5f985-7473-447b-a1f3-d886935ae0bc" (UID: "0ee5f985-7473-447b-a1f3-d886935ae0bc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:24:17.147047 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.147017    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-token" (OuterVolumeSpecName: "istio-token") pod "0ee5f985-7473-447b-a1f3-d886935ae0bc" (UID: "0ee5f985-7473-447b-a1f3-d886935ae0bc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:24:17.147175 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.147056    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0ee5f985-7473-447b-a1f3-d886935ae0bc" (UID: "0ee5f985-7473-447b-a1f3-d886935ae0bc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:24:17.147408 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.147389    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0ee5f985-7473-447b-a1f3-d886935ae0bc" (UID: "0ee5f985-7473-447b-a1f3-d886935ae0bc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:24:17.147478 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.147461    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-kube-api-access-h6zzw" (OuterVolumeSpecName: "kube-api-access-h6zzw") pod "0ee5f985-7473-447b-a1f3-d886935ae0bc" (UID: "0ee5f985-7473-447b-a1f3-d886935ae0bc"). InnerVolumeSpecName "kube-api-access-h6zzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:24:17.246271 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.246231    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-socket\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:17.246271 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.246268    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6zzw\" (UniqueName: \"kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-kube-api-access-h6zzw\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:17.246271 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.246278    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-token\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:17.246271 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.246286    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-envoy\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:17.246271 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.246294    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ee5f985-7473-447b-a1f3-d886935ae0bc-istio-podinfo\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:17.246271 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.246302    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ee5f985-7473-447b-a1f3-d886935ae0bc-workload-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:24:17.408889 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.408823    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"]
Sep 08 14:24:17.410501 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.410472    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e2a04d-openshift-default-74855dcccb-zmwnw"]
Sep 08 14:24:17.959185 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:24:17.959153    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" path="/var/lib/kubelet/pods/0ee5f985-7473-447b-a1f3-d886935ae0bc/volumes"
Sep 08 14:25:15.941215 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941174    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-tqmf9"]
Sep 08 14:25:15.941763 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941748    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2f2f32a-8fe4-4408-8b0a-0c100b1217fc" containerName="manager"
Sep 08 14:25:15.941839 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941769    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2f2f32a-8fe4-4408-8b0a-0c100b1217fc" containerName="manager"
Sep 08 14:25:15.941839 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941789    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8a373f0-9895-4b6c-b37b-e423ba7bf809" containerName="revoke-keys"
Sep 08 14:25:15.941839 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941798    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a373f0-9895-4b6c-b37b-e423ba7bf809" containerName="revoke-keys"
Sep 08 14:25:15.941839 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941815    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy"
Sep 08 14:25:15.941839 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941823    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy"
Sep 08 14:25:15.942144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941843    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy"
Sep 08 14:25:15.942144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941851    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy"
Sep 08 14:25:15.942144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941968    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e8781c7-053e-4317-b816-fc23d9c000da" containerName="istio-proxy"
Sep 08 14:25:15.942144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941985    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8a373f0-9895-4b6c-b37b-e423ba7bf809" containerName="revoke-keys"
Sep 08 14:25:15.942144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.941995    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy"
Sep 08 14:25:15.942144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.942005    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ee5f985-7473-447b-a1f3-d886935ae0bc" containerName="istio-proxy"
Sep 08 14:25:15.942144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.942015    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2f2f32a-8fe4-4408-8b0a-0c100b1217fc" containerName="manager"
Sep 08 14:25:15.945251 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.945230    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:15.947783 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.947766    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 14:25:15.948773 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.948748    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vj9d6\""
Sep 08 14:25:15.948937 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.948775    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 14:25:15.949033 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.948935    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 14:25:15.959700 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:15.959677    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-tqmf9"]
Sep 08 14:25:16.106408 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.106364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-metrics-certs\") pod \"maas-controller-6d9494c8c4-tqmf9\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") " pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:16.106596 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.106427    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-webhook-cert\") pod \"maas-controller-6d9494c8c4-tqmf9\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") " pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:16.106659 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.106600    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4j2p\" (UniqueName: \"kubernetes.io/projected/150c5e62-8270-4db1-b2c5-4ffea7244231-kube-api-access-l4j2p\") pod \"maas-controller-6d9494c8c4-tqmf9\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") " pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:16.207742 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.207656    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4j2p\" (UniqueName: \"kubernetes.io/projected/150c5e62-8270-4db1-b2c5-4ffea7244231-kube-api-access-l4j2p\") pod \"maas-controller-6d9494c8c4-tqmf9\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") " pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:16.207742 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.207717    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-metrics-certs\") pod \"maas-controller-6d9494c8c4-tqmf9\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") " pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:16.207961 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.207755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-webhook-cert\") pod \"maas-controller-6d9494c8c4-tqmf9\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") " pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:16.210520 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.210496    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-metrics-certs\") pod \"maas-controller-6d9494c8c4-tqmf9\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") " pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:16.210520 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.210508    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-webhook-cert\") pod \"maas-controller-6d9494c8c4-tqmf9\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") " pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:16.217484 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.217455    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4j2p\" (UniqueName: \"kubernetes.io/projected/150c5e62-8270-4db1-b2c5-4ffea7244231-kube-api-access-l4j2p\") pod \"maas-controller-6d9494c8c4-tqmf9\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") " pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:16.256017 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.255982    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:16.534405 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.534319    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-tqmf9"]
Sep 08 14:25:16.536746 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:25:16.536712    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150c5e62_8270_4db1_b2c5_4ffea7244231.slice/crio-dab15b6f5ad7031b1bc19fab0f8df2fdc52f6b63057d8643999ca7666e675080 WatchSource:0}: Error finding container dab15b6f5ad7031b1bc19fab0f8df2fdc52f6b63057d8643999ca7666e675080: Status 404 returned error can't find the container with id dab15b6f5ad7031b1bc19fab0f8df2fdc52f6b63057d8643999ca7666e675080
Sep 08 14:25:16.538225 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:16.538205    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:25:17.388791 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:17.388702    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9" event={"ID":"150c5e62-8270-4db1-b2c5-4ffea7244231","Type":"ContainerStarted","Data":"4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9"}
Sep 08 14:25:17.388791 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:17.388759    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9" event={"ID":"150c5e62-8270-4db1-b2c5-4ffea7244231","Type":"ContainerStarted","Data":"dab15b6f5ad7031b1bc19fab0f8df2fdc52f6b63057d8643999ca7666e675080"}
Sep 08 14:25:17.388791 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:17.388789    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:17.418485 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:17.418427    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9" podStartSLOduration=1.889319651 podStartE2EDuration="2.418408465s" totalImagesPullingTime="529.088814ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:25:17.417686539 +0000 UTC m=+1607.912657236" watchObservedRunningTime="2026-09-08 14:25:17.418408465 +0000 UTC m=+1607.913379176"
Sep 08 14:25:28.396558 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:28.396482    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:37.810980 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:37.810946    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-tqmf9"]
Sep 08 14:25:37.811551 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:37.811143    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9" podUID="150c5e62-8270-4db1-b2c5-4ffea7244231" containerName="manager" containerID="cri-o://4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9" gracePeriod=30
Sep 08 14:25:38.057732 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.057705    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:38.109379 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.109296    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-webhook-cert\") pod \"150c5e62-8270-4db1-b2c5-4ffea7244231\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") "
Sep 08 14:25:38.109379 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.109336    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-metrics-certs\") pod \"150c5e62-8270-4db1-b2c5-4ffea7244231\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") "
Sep 08 14:25:38.109587 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.109381    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4j2p\" (UniqueName: \"kubernetes.io/projected/150c5e62-8270-4db1-b2c5-4ffea7244231-kube-api-access-l4j2p\") pod \"150c5e62-8270-4db1-b2c5-4ffea7244231\" (UID: \"150c5e62-8270-4db1-b2c5-4ffea7244231\") "
Sep 08 14:25:38.111562 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.111527    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "150c5e62-8270-4db1-b2c5-4ffea7244231" (UID: "150c5e62-8270-4db1-b2c5-4ffea7244231"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:25:38.111688 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.111559    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "150c5e62-8270-4db1-b2c5-4ffea7244231" (UID: "150c5e62-8270-4db1-b2c5-4ffea7244231"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:25:38.111688 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.111632    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/150c5e62-8270-4db1-b2c5-4ffea7244231-kube-api-access-l4j2p" (OuterVolumeSpecName: "kube-api-access-l4j2p") pod "150c5e62-8270-4db1-b2c5-4ffea7244231" (UID: "150c5e62-8270-4db1-b2c5-4ffea7244231"). InnerVolumeSpecName "kube-api-access-l4j2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:25:38.210400 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.210358    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-webhook-cert\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:25:38.210400 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.210388    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150c5e62-8270-4db1-b2c5-4ffea7244231-metrics-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:25:38.210400 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.210403    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4j2p\" (UniqueName: \"kubernetes.io/projected/150c5e62-8270-4db1-b2c5-4ffea7244231-kube-api-access-l4j2p\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:25:38.497708 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.497672    2564 generic.go:356] "Generic (PLEG): container finished" podID="150c5e62-8270-4db1-b2c5-4ffea7244231" containerID="4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9" exitCode=0
Sep 08 14:25:38.497954 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.497726    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9"
Sep 08 14:25:38.497954 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.497776    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9" event={"ID":"150c5e62-8270-4db1-b2c5-4ffea7244231","Type":"ContainerDied","Data":"4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9"}
Sep 08 14:25:38.497954 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.497840    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9494c8c4-tqmf9" event={"ID":"150c5e62-8270-4db1-b2c5-4ffea7244231","Type":"ContainerDied","Data":"dab15b6f5ad7031b1bc19fab0f8df2fdc52f6b63057d8643999ca7666e675080"}
Sep 08 14:25:38.497954 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.497899    2564 scope.go:117] "RemoveContainer" containerID="4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9"
Sep 08 14:25:38.507182 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.507162    2564 scope.go:117] "RemoveContainer" containerID="4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9"
Sep 08 14:25:38.507475 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:25:38.507454    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9\": container with ID starting with 4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9 not found: ID does not exist" containerID="4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9"
Sep 08 14:25:38.507523 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.507487    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9"} err="failed to get container status \"4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9\": rpc error: code = NotFound desc = could not find container \"4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9\": container with ID starting with 4ad6347adcb35b413e4b930129f811b92bafb4febe5fdbeb05bb5ed1733446e9 not found: ID does not exist"
Sep 08 14:25:38.521431 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.521408    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-tqmf9"]
Sep 08 14:25:38.528026 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:38.528008    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-tqmf9"]
Sep 08 14:25:39.313503 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.313471    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-6kxgx"]
Sep 08 14:25:39.313903 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.313817    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="150c5e62-8270-4db1-b2c5-4ffea7244231" containerName="manager"
Sep 08 14:25:39.313903 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.313828    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="150c5e62-8270-4db1-b2c5-4ffea7244231" containerName="manager"
Sep 08 14:25:39.313986 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.313918    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="150c5e62-8270-4db1-b2c5-4ffea7244231" containerName="manager"
Sep 08 14:25:39.318211 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.318194    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:39.321930 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.321909    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 14:25:39.322065 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.321926    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 14:25:39.322065 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.321909    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 14:25:39.322065 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.321909    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vj9d6\""
Sep 08 14:25:39.328159 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.328136    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-6kxgx"]
Sep 08 14:25:39.421509 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.421474    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/45a8475c-8d50-4ed9-9e54-e201a0f15cb3-webhook-cert\") pod \"maas-controller-6d9494c8c4-6kxgx\" (UID: \"45a8475c-8d50-4ed9-9e54-e201a0f15cb3\") " pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:39.421697 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.421580    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45a8475c-8d50-4ed9-9e54-e201a0f15cb3-metrics-certs\") pod \"maas-controller-6d9494c8c4-6kxgx\" (UID: \"45a8475c-8d50-4ed9-9e54-e201a0f15cb3\") " pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:39.421697 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.421633    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqpnr\" (UniqueName: \"kubernetes.io/projected/45a8475c-8d50-4ed9-9e54-e201a0f15cb3-kube-api-access-zqpnr\") pod \"maas-controller-6d9494c8c4-6kxgx\" (UID: \"45a8475c-8d50-4ed9-9e54-e201a0f15cb3\") " pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:39.522515 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.522476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45a8475c-8d50-4ed9-9e54-e201a0f15cb3-metrics-certs\") pod \"maas-controller-6d9494c8c4-6kxgx\" (UID: \"45a8475c-8d50-4ed9-9e54-e201a0f15cb3\") " pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:39.522736 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.522559    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqpnr\" (UniqueName: \"kubernetes.io/projected/45a8475c-8d50-4ed9-9e54-e201a0f15cb3-kube-api-access-zqpnr\") pod \"maas-controller-6d9494c8c4-6kxgx\" (UID: \"45a8475c-8d50-4ed9-9e54-e201a0f15cb3\") " pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:39.522736 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.522612    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/45a8475c-8d50-4ed9-9e54-e201a0f15cb3-webhook-cert\") pod \"maas-controller-6d9494c8c4-6kxgx\" (UID: \"45a8475c-8d50-4ed9-9e54-e201a0f15cb3\") " pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:39.525122 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.525099    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45a8475c-8d50-4ed9-9e54-e201a0f15cb3-metrics-certs\") pod \"maas-controller-6d9494c8c4-6kxgx\" (UID: \"45a8475c-8d50-4ed9-9e54-e201a0f15cb3\") " pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:39.525238 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.525150    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/45a8475c-8d50-4ed9-9e54-e201a0f15cb3-webhook-cert\") pod \"maas-controller-6d9494c8c4-6kxgx\" (UID: \"45a8475c-8d50-4ed9-9e54-e201a0f15cb3\") " pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:39.530988 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.530962    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqpnr\" (UniqueName: \"kubernetes.io/projected/45a8475c-8d50-4ed9-9e54-e201a0f15cb3-kube-api-access-zqpnr\") pod \"maas-controller-6d9494c8c4-6kxgx\" (UID: \"45a8475c-8d50-4ed9-9e54-e201a0f15cb3\") " pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:39.629763 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.629671    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:39.904439 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.904414    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d9494c8c4-6kxgx"]
Sep 08 14:25:39.907205 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:25:39.907176    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45a8475c_8d50_4ed9_9e54_e201a0f15cb3.slice/crio-c678933fc95a68a80b1fb99ed7ca6c4dba7c41d7434d318af8c8a449aeadb321 WatchSource:0}: Error finding container c678933fc95a68a80b1fb99ed7ca6c4dba7c41d7434d318af8c8a449aeadb321: Status 404 returned error can't find the container with id c678933fc95a68a80b1fb99ed7ca6c4dba7c41d7434d318af8c8a449aeadb321
Sep 08 14:25:39.959044 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:39.959013    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="150c5e62-8270-4db1-b2c5-4ffea7244231" path="/var/lib/kubelet/pods/150c5e62-8270-4db1-b2c5-4ffea7244231/volumes"
Sep 08 14:25:40.509700 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:40.509656    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9494c8c4-6kxgx" event={"ID":"45a8475c-8d50-4ed9-9e54-e201a0f15cb3","Type":"ContainerStarted","Data":"613791cfbb57d56e4387b86295fa2503d45b51694f200b075ba1d7d2df0e6b1e"}
Sep 08 14:25:40.510144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:40.509724    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:25:40.510144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:40.509747    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d9494c8c4-6kxgx" event={"ID":"45a8475c-8d50-4ed9-9e54-e201a0f15cb3","Type":"ContainerStarted","Data":"c678933fc95a68a80b1fb99ed7ca6c4dba7c41d7434d318af8c8a449aeadb321"}
Sep 08 14:25:40.529368 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:40.529315    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d9494c8c4-6kxgx" podStartSLOduration=1.133474672 podStartE2EDuration="1.529297395s" totalImagesPullingTime="395.822723ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:25:40.527522756 +0000 UTC m=+1631.022493445" watchObservedRunningTime="2026-09-08 14:25:40.529297395 +0000 UTC m=+1631.024268126"
Sep 08 14:25:51.517606 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:25:51.517570    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d9494c8c4-6kxgx"
Sep 08 14:26:30.343192 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:26:30.343156    2564 scope.go:117] "RemoveContainer" containerID="a1919244399668da86574643904b5d3b5a25a251ee74ac54d0bd8ef0ba9c4e69"
Sep 08 14:28:33.926948 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:28:33.926922    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:28:33.937550 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:28:33.937526    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:29:30.379614 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:30.379579    2564 scope.go:117] "RemoveContainer" containerID="972962a11cb27fc9620bf7165f01089d43bf53d7ead919abdb221d70225176f7"
Sep 08 14:29:30.390609 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:30.389015    2564 scope.go:117] "RemoveContainer" containerID="4cf7e4e84b27a521458f4993bf273fdc639273db1a1377d959e3200376000141"
Sep 08 14:29:54.771024 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.770927    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"]
Sep 08 14:29:54.774584 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.774563    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.777250 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.777221    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-53815519-kserve-self-signed-certs\""
Sep 08 14:29:54.785289 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.785266    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"]
Sep 08 14:29:54.815634 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.815603    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-dshm\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.815773 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.815644    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73ea79d4-9397-4f37-ba66-72c52927e4a7-tls-certs\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.815773 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.815755    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-home\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.815922 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.815781    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcqkx\" (UniqueName: \"kubernetes.io/projected/73ea79d4-9397-4f37-ba66-72c52927e4a7-kube-api-access-rcqkx\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.815922 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.815832    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-tmp-dir\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.815922 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.815884    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-model-cache\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.916792 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.916759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-home\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.917011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.916798    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcqkx\" (UniqueName: \"kubernetes.io/projected/73ea79d4-9397-4f37-ba66-72c52927e4a7-kube-api-access-rcqkx\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.917011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.916846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-tmp-dir\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.917011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.916910    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-model-cache\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.917011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.916948    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-dshm\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.917011 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.916979    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73ea79d4-9397-4f37-ba66-72c52927e4a7-tls-certs\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.917280 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.917258    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-home\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.917340 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.917316    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-tmp-dir\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.917375 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.917327    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-model-cache\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.919468 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.919441    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-dshm\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.919716 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.919687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73ea79d4-9397-4f37-ba66-72c52927e4a7-tls-certs\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:54.927272 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:54.927240    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcqkx\" (UniqueName: \"kubernetes.io/projected/73ea79d4-9397-4f37-ba66-72c52927e4a7-kube-api-access-rcqkx\") pod \"e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") " pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:55.085049 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:55.084964    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:29:55.358344 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:55.358319    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"]
Sep 08 14:29:55.360252 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:29:55.360221    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73ea79d4_9397_4f37_ba66_72c52927e4a7.slice/crio-9f13f2dc2a98dfd3715815edf56da9f3366c2534efef87e2dfb8161ffe910389 WatchSource:0}: Error finding container 9f13f2dc2a98dfd3715815edf56da9f3366c2534efef87e2dfb8161ffe910389: Status 404 returned error can't find the container with id 9f13f2dc2a98dfd3715815edf56da9f3366c2534efef87e2dfb8161ffe910389
Sep 08 14:29:55.735544 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:55.735504    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv" event={"ID":"73ea79d4-9397-4f37-ba66-72c52927e4a7","Type":"ContainerStarted","Data":"63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996"}
Sep 08 14:29:55.735756 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:55.735557    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv" event={"ID":"73ea79d4-9397-4f37-ba66-72c52927e4a7","Type":"ContainerStarted","Data":"9f13f2dc2a98dfd3715815edf56da9f3366c2534efef87e2dfb8161ffe910389"}
Sep 08 14:29:55.752771 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:55.752729    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv" podStartSLOduration=1.752715246 podStartE2EDuration="1.752715246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:29:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:29:55.751399239 +0000 UTC m=+1886.246369924" watchObservedRunningTime="2026-09-08 14:29:55.752715246 +0000 UTC m=+1886.247685930"
Sep 08 14:29:56.085268 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:29:56.085169    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:30:00.133555 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:00.133524    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7"]
Sep 08 14:30:00.137126 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:00.137109    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7"
Sep 08 14:30:00.139687 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:00.139669    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d2tr9\""
Sep 08 14:30:00.142996 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:00.142970    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7"]
Sep 08 14:30:00.149563 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:00.149541    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7"
Sep 08 14:30:00.422339 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:00.422315    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7"]
Sep 08 14:30:00.424736 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:30:00.424708    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf628a8f0_9183_4f54_ab2e_fdd3c9561d73.slice/crio-139a7b23c4cbc62d2e87d620803ce9606cf443d8278d8960dbe81c435bdcf165 WatchSource:0}: Error finding container 139a7b23c4cbc62d2e87d620803ce9606cf443d8278d8960dbe81c435bdcf165: Status 404 returned error can't find the container with id 139a7b23c4cbc62d2e87d620803ce9606cf443d8278d8960dbe81c435bdcf165
Sep 08 14:30:00.761043 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:00.761009    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7" event={"ID":"f628a8f0-9183-4f54-ab2e-fdd3c9561d73","Type":"ContainerStarted","Data":"0cb904b827a6dc6781880202c87be8fc27853b6ead1cfeb974952a4085948439"}
Sep 08 14:30:00.761188 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:00.761069    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7" event={"ID":"f628a8f0-9183-4f54-ab2e-fdd3c9561d73","Type":"ContainerStarted","Data":"139a7b23c4cbc62d2e87d620803ce9606cf443d8278d8960dbe81c435bdcf165"}
Sep 08 14:30:00.777211 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:00.777157    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7" podStartSLOduration=0.777141848 podStartE2EDuration="777.141848ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:30:00.776251842 +0000 UTC m=+1891.271222539" watchObservedRunningTime="2026-09-08 14:30:00.777141848 +0000 UTC m=+1891.272112571"
Sep 08 14:30:01.767852 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:01.767814    2564 generic.go:356] "Generic (PLEG): container finished" podID="f628a8f0-9183-4f54-ab2e-fdd3c9561d73" containerID="0cb904b827a6dc6781880202c87be8fc27853b6ead1cfeb974952a4085948439" exitCode=0
Sep 08 14:30:01.768272 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:01.767900    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7" event={"ID":"f628a8f0-9183-4f54-ab2e-fdd3c9561d73","Type":"ContainerDied","Data":"0cb904b827a6dc6781880202c87be8fc27853b6ead1cfeb974952a4085948439"}
Sep 08 14:30:02.908510 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:02.908483    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7"
Sep 08 14:30:03.780017 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:03.779980    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7" event={"ID":"f628a8f0-9183-4f54-ab2e-fdd3c9561d73","Type":"ContainerDied","Data":"139a7b23c4cbc62d2e87d620803ce9606cf443d8278d8960dbe81c435bdcf165"}
Sep 08 14:30:03.780203 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:03.780033    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="139a7b23c4cbc62d2e87d620803ce9606cf443d8278d8960dbe81c435bdcf165"
Sep 08 14:30:03.780203 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:03.779989    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814630-7dkx7"
Sep 08 14:30:05.086144 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:05.086103    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:30:05.099162 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:05.099134    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:30:05.801054 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:05.801020    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:30:21.990813 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:21.990775    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"]
Sep 08 14:30:30.411459 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:30.411421    2564 scope.go:117] "RemoveContainer" containerID="0463bfb015950497b81584448964c6f008f8d50c1ddcc76634b4ee95074a180a"
Sep 08 14:30:37.010987 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.010945    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv" podUID="73ea79d4-9397-4f37-ba66-72c52927e4a7" containerName="main" containerID="cri-o://63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996" gracePeriod=45
Sep 08 14:30:37.259342 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.259316    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:30:37.317844 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.317756    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73ea79d4-9397-4f37-ba66-72c52927e4a7-tls-certs\") pod \"73ea79d4-9397-4f37-ba66-72c52927e4a7\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") "
Sep 08 14:30:37.317844 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.317806    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-tmp-dir\") pod \"73ea79d4-9397-4f37-ba66-72c52927e4a7\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") "
Sep 08 14:30:37.318084 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.317848    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-dshm\") pod \"73ea79d4-9397-4f37-ba66-72c52927e4a7\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") "
Sep 08 14:30:37.318084 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.317919    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-home\") pod \"73ea79d4-9397-4f37-ba66-72c52927e4a7\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") "
Sep 08 14:30:37.318084 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.317969    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-model-cache\") pod \"73ea79d4-9397-4f37-ba66-72c52927e4a7\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") "
Sep 08 14:30:37.318084 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.318002    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcqkx\" (UniqueName: \"kubernetes.io/projected/73ea79d4-9397-4f37-ba66-72c52927e4a7-kube-api-access-rcqkx\") pod \"73ea79d4-9397-4f37-ba66-72c52927e4a7\" (UID: \"73ea79d4-9397-4f37-ba66-72c52927e4a7\") "
Sep 08 14:30:37.318268 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.318077    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "73ea79d4-9397-4f37-ba66-72c52927e4a7" (UID: "73ea79d4-9397-4f37-ba66-72c52927e4a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:30:37.318268 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.318183    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-home" (OuterVolumeSpecName: "home") pod "73ea79d4-9397-4f37-ba66-72c52927e4a7" (UID: "73ea79d4-9397-4f37-ba66-72c52927e4a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:30:37.318268 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.318257    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-model-cache" (OuterVolumeSpecName: "model-cache") pod "73ea79d4-9397-4f37-ba66-72c52927e4a7" (UID: "73ea79d4-9397-4f37-ba66-72c52927e4a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:30:37.318403 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.318340    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-model-cache\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:30:37.318403 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.318354    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-tmp-dir\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:30:37.318403 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.318362    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-home\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:30:37.320179 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.320153    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73ea79d4-9397-4f37-ba66-72c52927e4a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "73ea79d4-9397-4f37-ba66-72c52927e4a7" (UID: "73ea79d4-9397-4f37-ba66-72c52927e4a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:30:37.320179 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.320165    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73ea79d4-9397-4f37-ba66-72c52927e4a7-kube-api-access-rcqkx" (OuterVolumeSpecName: "kube-api-access-rcqkx") pod "73ea79d4-9397-4f37-ba66-72c52927e4a7" (UID: "73ea79d4-9397-4f37-ba66-72c52927e4a7"). InnerVolumeSpecName "kube-api-access-rcqkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:30:37.320340 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.320204    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-dshm" (OuterVolumeSpecName: "dshm") pod "73ea79d4-9397-4f37-ba66-72c52927e4a7" (UID: "73ea79d4-9397-4f37-ba66-72c52927e4a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:30:37.419171 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.419138    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73ea79d4-9397-4f37-ba66-72c52927e4a7-dshm\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:30:37.419171 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.419164    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcqkx\" (UniqueName: \"kubernetes.io/projected/73ea79d4-9397-4f37-ba66-72c52927e4a7-kube-api-access-rcqkx\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:30:37.419171 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.419175    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73ea79d4-9397-4f37-ba66-72c52927e4a7-tls-certs\") on node \"ip-10-0-133-79.ec2.internal\" DevicePath \"\""
Sep 08 14:30:37.951294 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.951257    2564 generic.go:356] "Generic (PLEG): container finished" podID="73ea79d4-9397-4f37-ba66-72c52927e4a7" containerID="63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996" exitCode=0
Sep 08 14:30:37.951294 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.951295    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv" event={"ID":"73ea79d4-9397-4f37-ba66-72c52927e4a7","Type":"ContainerDied","Data":"63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996"}
Sep 08 14:30:37.951525 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.951295    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"
Sep 08 14:30:37.951525 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.951340    2564 scope.go:117] "RemoveContainer" containerID="63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996"
Sep 08 14:30:37.951525 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.951326    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv" event={"ID":"73ea79d4-9397-4f37-ba66-72c52927e4a7","Type":"ContainerDied","Data":"9f13f2dc2a98dfd3715815edf56da9f3366c2534efef87e2dfb8161ffe910389"}
Sep 08 14:30:37.961190 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.961170    2564 scope.go:117] "RemoveContainer" containerID="63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996"
Sep 08 14:30:37.961460 ip-10-0-133-79 kubenswrapper[2564]: E0908 14:30:37.961445    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996\": container with ID starting with 63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996 not found: ID does not exist" containerID="63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996"
Sep 08 14:30:37.961507 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.961467    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996"} err="failed to get container status \"63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996\": rpc error: code = NotFound desc = could not find container \"63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996\": container with ID starting with 63080c75845fb9720d7982cdb0440913781c5e4cf34b398ce12c1b2c4dc75996 not found: ID does not exist"
Sep 08 14:30:37.974597 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.974576    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"]
Sep 08 14:30:37.980512 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:37.980490    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-53815519-kserve-6cbd8cc85b-sx6qv"]
Sep 08 14:30:39.958819 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:30:39.958782    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73ea79d4-9397-4f37-ba66-72c52927e4a7" path="/var/lib/kubelet/pods/73ea79d4-9397-4f37-ba66-72c52927e4a7/volumes"
Sep 08 14:31:46.656824 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:46.656795    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-7d548556bc-8rrsf_e03268be-cb29-4268-895d-1b91e5e17919/manager/0.log"
Sep 08 14:31:46.768735 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:46.768701    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6d9494c8c4-6kxgx_45a8475c-8d50-4ed9-9e54-e201a0f15cb3/manager/0.log"
Sep 08 14:31:47.037278 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:47.037251    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-bb7b75cc4-k9sn2_99cb8a91-5ddb-4e51-896c-769517de4696/manager/0.log"
Sep 08 14:31:47.712741 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:47.712712    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29814630-7dkx7_f628a8f0-9183-4f54-ab2e-fdd3c9561d73/cleanup/0.log"
Sep 08 14:31:48.943230 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:48.943202    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-857td_db4d6ec1-55a6-4913-9387-e2ed8bf28c69/kuadrant-console-plugin/0.log"
Sep 08 14:31:50.024051 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:50.024028    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f4c5d6b6-rrbxj_ad87da59-0feb-4378-a627-f2a2db2e6975/kube-auth-proxy/0.log"
Sep 08 14:31:51.162611 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:51.162576    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-698db7d589-qm6px_26706892-e4b5-45b4-b087-61802acfbfc2/main/0.log"
Sep 08 14:31:51.390752 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:51.390717    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-c68ddf6l6zt_605c5174-b52a-4d85-80f3-1e8fa6bc1f78/main/0.log"
Sep 08 14:31:56.715657 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.715610    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jjr46/must-gather-9lvqp"]
Sep 08 14:31:56.716279 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.716021    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73ea79d4-9397-4f37-ba66-72c52927e4a7" containerName="main"
Sep 08 14:31:56.716279 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.716035    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="73ea79d4-9397-4f37-ba66-72c52927e4a7" containerName="main"
Sep 08 14:31:56.716279 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.716047    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f628a8f0-9183-4f54-ab2e-fdd3c9561d73" containerName="cleanup"
Sep 08 14:31:56.716279 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.716053    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f628a8f0-9183-4f54-ab2e-fdd3c9561d73" containerName="cleanup"
Sep 08 14:31:56.716279 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.716121    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="73ea79d4-9397-4f37-ba66-72c52927e4a7" containerName="main"
Sep 08 14:31:56.716279 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.716131    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f628a8f0-9183-4f54-ab2e-fdd3c9561d73" containerName="cleanup"
Sep 08 14:31:56.719325 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.719303    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jjr46/must-gather-9lvqp"
Sep 08 14:31:56.722249 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.722229    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jjr46\"/\"kube-root-ca.crt\""
Sep 08 14:31:56.722352 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.722251    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jjr46\"/\"openshift-service-ca.crt\""
Sep 08 14:31:56.722568 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.722554    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jjr46\"/\"default-dockercfg-trvv4\""
Sep 08 14:31:56.735009 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.734987    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jjr46/must-gather-9lvqp"]
Sep 08 14:31:56.754955 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.754922    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7e7157f4-7e9d-4907-94ec-4d11a4600123-must-gather-output\") pod \"must-gather-9lvqp\" (UID: \"7e7157f4-7e9d-4907-94ec-4d11a4600123\") " pod="openshift-must-gather-jjr46/must-gather-9lvqp"
Sep 08 14:31:56.755103 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.754988    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlpsw\" (UniqueName: \"kubernetes.io/projected/7e7157f4-7e9d-4907-94ec-4d11a4600123-kube-api-access-jlpsw\") pod \"must-gather-9lvqp\" (UID: \"7e7157f4-7e9d-4907-94ec-4d11a4600123\") " pod="openshift-must-gather-jjr46/must-gather-9lvqp"
Sep 08 14:31:56.856260 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.856215    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlpsw\" (UniqueName: \"kubernetes.io/projected/7e7157f4-7e9d-4907-94ec-4d11a4600123-kube-api-access-jlpsw\") pod \"must-gather-9lvqp\" (UID: \"7e7157f4-7e9d-4907-94ec-4d11a4600123\") " pod="openshift-must-gather-jjr46/must-gather-9lvqp"
Sep 08 14:31:56.856454 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.856320    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7e7157f4-7e9d-4907-94ec-4d11a4600123-must-gather-output\") pod \"must-gather-9lvqp\" (UID: \"7e7157f4-7e9d-4907-94ec-4d11a4600123\") " pod="openshift-must-gather-jjr46/must-gather-9lvqp"
Sep 08 14:31:56.856597 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.856580    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7e7157f4-7e9d-4907-94ec-4d11a4600123-must-gather-output\") pod \"must-gather-9lvqp\" (UID: \"7e7157f4-7e9d-4907-94ec-4d11a4600123\") " pod="openshift-must-gather-jjr46/must-gather-9lvqp"
Sep 08 14:31:56.864704 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:56.864675    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlpsw\" (UniqueName: \"kubernetes.io/projected/7e7157f4-7e9d-4907-94ec-4d11a4600123-kube-api-access-jlpsw\") pod \"must-gather-9lvqp\" (UID: \"7e7157f4-7e9d-4907-94ec-4d11a4600123\") " pod="openshift-must-gather-jjr46/must-gather-9lvqp"
Sep 08 14:31:57.029037 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:57.028958    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jjr46/must-gather-9lvqp"
Sep 08 14:31:57.303846 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:57.303822    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jjr46/must-gather-9lvqp"]
Sep 08 14:31:57.306364 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:31:57.306327    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e7157f4_7e9d_4907_94ec_4d11a4600123.slice/crio-4fd9201edc53cf75615d751dd9502ceb8e2ff7a4d55782f34feea70e151d5655 WatchSource:0}: Error finding container 4fd9201edc53cf75615d751dd9502ceb8e2ff7a4d55782f34feea70e151d5655: Status 404 returned error can't find the container with id 4fd9201edc53cf75615d751dd9502ceb8e2ff7a4d55782f34feea70e151d5655
Sep 08 14:31:57.308537 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:57.308521    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:31:57.335273 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:31:57.335231    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jjr46/must-gather-9lvqp" event={"ID":"7e7157f4-7e9d-4907-94ec-4d11a4600123","Type":"ContainerStarted","Data":"4fd9201edc53cf75615d751dd9502ceb8e2ff7a4d55782f34feea70e151d5655"}
Sep 08 14:32:01.367280 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:01.367241    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jjr46/must-gather-9lvqp" event={"ID":"7e7157f4-7e9d-4907-94ec-4d11a4600123","Type":"ContainerStarted","Data":"3b4e57e8f6530201841e105c3a79d85c6c663b367fe35ff27767bad16b23842e"}
Sep 08 14:32:01.367676 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:01.367291    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jjr46/must-gather-9lvqp" event={"ID":"7e7157f4-7e9d-4907-94ec-4d11a4600123","Type":"ContainerStarted","Data":"df7b3f601aae1855aaa328e8c333ab726b69649d19503807236fe4e6e47e1896"}
Sep 08 14:32:01.386785 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:01.386730    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jjr46/must-gather-9lvqp" podStartSLOduration=1.969507866 podStartE2EDuration="5.38671463s" totalImagesPullingTime="3.417206764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:31:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:32:01.385037613 +0000 UTC m=+2011.880008298" watchObservedRunningTime="2026-09-08 14:32:01.38671463 +0000 UTC m=+2011.881685313"
Sep 08 14:32:02.380555 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:02.380526    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lfxpp_c2b891b6-0ff7-4267-b030-ee95f25a0727/global-pull-secret-syncer/0.log"
Sep 08 14:32:02.536472 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:02.536438    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g8grv_e10f70fd-1430-4819-b75b-a236e679fb6f/konnectivity-agent/0.log"
Sep 08 14:32:02.597110 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:02.597076    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-79.ec2.internal_24342c83cc108d3ad0b5e1eda2e8279a/haproxy/0.log"
Sep 08 14:32:06.916481 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:06.916452    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-857td_db4d6ec1-55a6-4913-9387-e2ed8bf28c69/kuadrant-console-plugin/0.log"
Sep 08 14:32:08.848136 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:08.848050    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-8xxr2_05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634/kube-state-metrics/0.log"
Sep 08 14:32:08.867395 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:08.867368    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-8xxr2_05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634/kube-rbac-proxy-main/0.log"
Sep 08 14:32:08.895805 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:08.895776    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-8xxr2_05aecc9a-b6bc-4e7e-81ef-7ec2b4ae1634/kube-rbac-proxy-self/0.log"
Sep 08 14:32:08.924302 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:08.924269    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-78d9cb5d8-76zfh_e702cd93-5a9c-4341-bcab-a74db83a2016/metrics-server/0.log"
Sep 08 14:32:09.059263 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.059233    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fr5t8_fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c/node-exporter/0.log"
Sep 08 14:32:09.080321 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.080298    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fr5t8_fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c/kube-rbac-proxy/0.log"
Sep 08 14:32:09.106176 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.106097    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fr5t8_fc89bfbe-e04e-4f5e-b2c6-ceda1b7a6d7c/init-textfile/0.log"
Sep 08 14:32:09.228743 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.228504    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-22mzn_0d9c7549-ca97-450c-9de8-9caf01336dbf/kube-rbac-proxy-main/0.log"
Sep 08 14:32:09.267249 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.267218    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-22mzn_0d9c7549-ca97-450c-9de8-9caf01336dbf/kube-rbac-proxy-self/0.log"
Sep 08 14:32:09.291125 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.291097    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-22mzn_0d9c7549-ca97-450c-9de8-9caf01336dbf/openshift-state-metrics/0.log"
Sep 08 14:32:09.361463 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.361385    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82b894e0-d6ef-485f-8e7d-20dc049ac702/prometheus/0.log"
Sep 08 14:32:09.440366 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.440338    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82b894e0-d6ef-485f-8e7d-20dc049ac702/config-reloader/0.log"
Sep 08 14:32:09.469180 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.469156    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82b894e0-d6ef-485f-8e7d-20dc049ac702/thanos-sidecar/0.log"
Sep 08 14:32:09.498396 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.498363    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82b894e0-d6ef-485f-8e7d-20dc049ac702/kube-rbac-proxy-web/0.log"
Sep 08 14:32:09.533380 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.533347    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82b894e0-d6ef-485f-8e7d-20dc049ac702/kube-rbac-proxy/0.log"
Sep 08 14:32:09.573070 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.573039    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82b894e0-d6ef-485f-8e7d-20dc049ac702/kube-rbac-proxy-thanos/0.log"
Sep 08 14:32:09.604008 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.603977    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82b894e0-d6ef-485f-8e7d-20dc049ac702/init-config-reloader/0.log"
Sep 08 14:32:09.740681 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.740639    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6456bdbb5d-m242q_90c0192f-f1a3-4417-b3e9-b97b41744f50/telemeter-client/0.log"
Sep 08 14:32:09.771273 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.771240    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6456bdbb5d-m242q_90c0192f-f1a3-4417-b3e9-b97b41744f50/reload/0.log"
Sep 08 14:32:09.802027 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:09.801999    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6456bdbb5d-m242q_90c0192f-f1a3-4417-b3e9-b97b41744f50/kube-rbac-proxy/0.log"
Sep 08 14:32:12.239815 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:12.239778    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/1.log"
Sep 08 14:32:12.246714 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:12.246685    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-7s6g6_7c2f5dad-d885-4865-84ba-9a31338284b5/console-operator/2.log"
Sep 08 14:32:14.540381 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:14.540349    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fwk9m_a313f4b1-b29f-4d5f-a580-5ed5bd99696d/dns/0.log"
Sep 08 14:32:14.563416 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:14.563392    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fwk9m_a313f4b1-b29f-4d5f-a580-5ed5bd99696d/kube-rbac-proxy/0.log"
Sep 08 14:32:14.639520 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:14.639492    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qgvf6_dbfe671c-f26a-484b-8e4a-d634b1731f98/dns-node-resolver/0.log"
Sep 08 14:32:14.929402 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:14.929305    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"]
Sep 08 14:32:14.940614 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:14.939597    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:14.942128 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:14.942097    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"]
Sep 08 14:32:15.044403 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.044372    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-lib-modules\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.044403 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.044408    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-sys\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.044647 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.044624    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88t5n\" (UniqueName: \"kubernetes.io/projected/6afa6825-3460-4edf-8cfb-a334c6f03b25-kube-api-access-88t5n\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.044740 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.044724    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-podres\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.044833 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.044807    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-proc\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.146398 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.146351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-podres\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.146592 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.146442    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-proc\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.146592 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.146501    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-lib-modules\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.146592 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.146528    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-sys\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.146592 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.146556    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-proc\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.146592 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.146552    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-podres\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.146592 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.146589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88t5n\" (UniqueName: \"kubernetes.io/projected/6afa6825-3460-4edf-8cfb-a334c6f03b25-kube-api-access-88t5n\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.146841 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.146623    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-sys\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.146841 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.146645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6afa6825-3460-4edf-8cfb-a334c6f03b25-lib-modules\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.155837 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.155804    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88t5n\" (UniqueName: \"kubernetes.io/projected/6afa6825-3460-4edf-8cfb-a334c6f03b25-kube-api-access-88t5n\") pod \"perf-node-gather-daemonset-shzsf\" (UID: \"6afa6825-3460-4edf-8cfb-a334c6f03b25\") " pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.256493 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.256459    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:15.339363 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.338883    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-47jpr_da52533e-1455-4ff1-8942-6cf7633a85a1/node-ca/0.log"
Sep 08 14:32:15.544449 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:15.544410    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"]
Sep 08 14:32:15.547653 ip-10-0-133-79 kubenswrapper[2564]: W0908 14:32:15.547623    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6afa6825_3460_4edf_8cfb_a334c6f03b25.slice/crio-2f8c7a38e10de2a2817a38836d37e7d35b09f7c412bac3d40f3ec7bf136e9d71 WatchSource:0}: Error finding container 2f8c7a38e10de2a2817a38836d37e7d35b09f7c412bac3d40f3ec7bf136e9d71: Status 404 returned error can't find the container with id 2f8c7a38e10de2a2817a38836d37e7d35b09f7c412bac3d40f3ec7bf136e9d71
Sep 08 14:32:16.473664 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:16.473598    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf" event={"ID":"6afa6825-3460-4edf-8cfb-a334c6f03b25","Type":"ContainerStarted","Data":"545cc3c02741844e2b251d7fc2cf98d14a0c6feb07e6d82f263197ccd916f182"}
Sep 08 14:32:16.473890 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:16.473696    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf" event={"ID":"6afa6825-3460-4edf-8cfb-a334c6f03b25","Type":"ContainerStarted","Data":"2f8c7a38e10de2a2817a38836d37e7d35b09f7c412bac3d40f3ec7bf136e9d71"}
Sep 08 14:32:16.473890 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:16.473735    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:16.496646 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:16.496595    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf" podStartSLOduration=2.4965780779999998 podStartE2EDuration="2.496578078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:32:16.495121408 +0000 UTC m=+2026.990092120" watchObservedRunningTime="2026-09-08 14:32:16.496578078 +0000 UTC m=+2026.991548764"
Sep 08 14:32:16.673135 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:16.673101    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f4c5d6b6-rrbxj_ad87da59-0feb-4378-a627-f2a2db2e6975/kube-auth-proxy/0.log"
Sep 08 14:32:17.748591 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:17.748557    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qsrr6_d4066c24-5beb-4011-a94b-cf594601f554/serve-healthcheck-canary/0.log"
Sep 08 14:32:18.412425 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:18.412390    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5n7f_7c39d180-d8f0-4b18-b7ff-70563ea2e867/kube-rbac-proxy/0.log"
Sep 08 14:32:18.435242 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:18.435211    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5n7f_7c39d180-d8f0-4b18-b7ff-70563ea2e867/exporter/0.log"
Sep 08 14:32:18.458757 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:18.458724    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5n7f_7c39d180-d8f0-4b18-b7ff-70563ea2e867/extractor/0.log"
Sep 08 14:32:20.771504 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:20.771468    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-77cd5f6c85-k5gsc_0eda0a01-7646-47f1-a4d5-fda4f1248de9/manager/0.log"
Sep 08 14:32:20.792254 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:20.792228    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-77cd5f6c85-k5gsc_0eda0a01-7646-47f1-a4d5-fda4f1248de9/copy-manifests/0.log"
Sep 08 14:32:21.951116 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:21.951048    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-controller-manager-c64769648-gxrfh_f129a405-b420-47b4-bb75-4bdc5c5580b6/manager/0.log"
Sep 08 14:32:22.202675 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:22.202591    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f4c5d6b6-rrbxj_ad87da59-0feb-4378-a627-f2a2db2e6975/kube-auth-proxy/0.log"
Sep 08 14:32:22.493511 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:22.493483    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jjr46/perf-node-gather-daemonset-shzsf"
Sep 08 14:32:22.761278 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:22.761192    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-7d548556bc-8rrsf_e03268be-cb29-4268-895d-1b91e5e17919/manager/0.log"
Sep 08 14:32:22.814978 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:22.814929    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6d9494c8c4-6kxgx_45a8475c-8d50-4ed9-9e54-e201a0f15cb3/manager/0.log"
Sep 08 14:32:22.910064 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:22.910026    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-bb7b75cc4-k9sn2_99cb8a91-5ddb-4e51-896c-769517de4696/manager/0.log"
Sep 08 14:32:24.097305 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:24.097268    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58f4c5d6b6-rrbxj_ad87da59-0feb-4378-a627-f2a2db2e6975/kube-auth-proxy/0.log"
Sep 08 14:32:30.025321 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:30.025289    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhsp9_05477ef7-e414-40b8-9d2d-b7da68fd264f/kube-multus-additional-cni-plugins/0.log"
Sep 08 14:32:30.049221 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:30.049197    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhsp9_05477ef7-e414-40b8-9d2d-b7da68fd264f/egress-router-binary-copy/0.log"
Sep 08 14:32:30.071012 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:30.070983    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhsp9_05477ef7-e414-40b8-9d2d-b7da68fd264f/cni-plugins/0.log"
Sep 08 14:32:30.093364 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:30.093340    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhsp9_05477ef7-e414-40b8-9d2d-b7da68fd264f/bond-cni-plugin/0.log"
Sep 08 14:32:30.116299 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:30.116274    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhsp9_05477ef7-e414-40b8-9d2d-b7da68fd264f/routeoverride-cni/0.log"
Sep 08 14:32:30.141911 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:30.141882    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhsp9_05477ef7-e414-40b8-9d2d-b7da68fd264f/whereabouts-cni-bincopy/0.log"
Sep 08 14:32:30.161943 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:30.161912    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nhsp9_05477ef7-e414-40b8-9d2d-b7da68fd264f/whereabouts-cni/0.log"
Sep 08 14:32:30.207139 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:30.207109    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gvwgp_f7869bcf-f35e-43bf-baa4-04fe6606396a/kube-multus/0.log"
Sep 08 14:32:30.394505 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:30.394432    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zkd7c_a9fa416b-05ee-4410-a741-c433bde86ee0/network-metrics-daemon/0.log"
Sep 08 14:32:30.412527 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:30.412503    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zkd7c_a9fa416b-05ee-4410-a741-c433bde86ee0/kube-rbac-proxy/0.log"
Sep 08 14:32:31.691735 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:31.691706    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zsjl2_8d73c1ac-fd9b-4432-a67f-1ce29ef0066d/ovn-controller/0.log"
Sep 08 14:32:31.710351 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:31.710324    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zsjl2_8d73c1ac-fd9b-4432-a67f-1ce29ef0066d/ovn-acl-logging/0.log"
Sep 08 14:32:31.734210 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:31.734162    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zsjl2_8d73c1ac-fd9b-4432-a67f-1ce29ef0066d/kube-rbac-proxy-node/0.log"
Sep 08 14:32:31.763029 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:31.763004    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zsjl2_8d73c1ac-fd9b-4432-a67f-1ce29ef0066d/kube-rbac-proxy-ovn-metrics/0.log"
Sep 08 14:32:31.782403 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:31.782381    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zsjl2_8d73c1ac-fd9b-4432-a67f-1ce29ef0066d/northd/0.log"
Sep 08 14:32:31.809528 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:31.809495    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zsjl2_8d73c1ac-fd9b-4432-a67f-1ce29ef0066d/nbdb/0.log"
Sep 08 14:32:31.831046 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:31.831016    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zsjl2_8d73c1ac-fd9b-4432-a67f-1ce29ef0066d/sbdb/0.log"
Sep 08 14:32:31.888436 ip-10-0-133-79 kubenswrapper[2564]: I0908 14:32:31.888410    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zsjl2_8d73c1ac-fd9b-4432-a67f-1ce29ef0066d/ovnkube-controller/0.log"