Jul 17 09:18:35.815766 ip-10-0-138-208 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 09:18:37.160849 ip-10-0-138-208 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:18:37.160849 ip-10-0-138-208 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 09:18:37.160849 ip-10-0-138-208 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:18:37.160849 ip-10-0-138-208 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 09:18:37.160849 ip-10-0-138-208 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:18:37.160849 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.155717    2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163431    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163448    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163451    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163455    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163459    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163464    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163468    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163470    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163473    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163476    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163479    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163482    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163485    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163488    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163491    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163494    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163497    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163500    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:37.164431 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163508    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163514    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163518    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163521    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163524    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163527    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163530    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163534    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163552    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163555    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163559    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163561    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163564    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163567    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163571    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163574    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163577    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163580    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163583    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163586    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:37.168557 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163589    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163592    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163595    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163598    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163601    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163604    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163607    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163610    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163613    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163615    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163618    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163621    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163624    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163627    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163630    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163633    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163636    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163638    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163641    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163644    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:37.169551 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163648    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163651    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163654    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163657    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163660    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163663    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163666    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163669    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163672    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163676    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163679    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163682    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163685    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163688    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163690    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163693    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163696    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163699    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163702    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163705    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:37.172805 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163708    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163711    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163714    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163717    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163720    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163724    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163727    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163730    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163735    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163739    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163742    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163745    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163748    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163751    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163754    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163759    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163762    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163765    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163769    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:37.173794 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163772    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.163775    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164572    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164581    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164584    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164588    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164591    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164594    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164597    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164600    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164603    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164606    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164609    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164612    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164616    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164621    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164624    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164627    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164630    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164634    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:37.178347 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164637    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164640    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164643    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164645    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164648    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164651    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164654    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164656    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164659    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164662    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164678    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164683    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164686    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164689    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164692    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164695    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164698    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164701    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164704    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164706    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:37.179355 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164709    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164712    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164715    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164719    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164722    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164724    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164727    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164730    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164733    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164745    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164749    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164752    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164757    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164760    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164763    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164766    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164769    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164772    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164774    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164777    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:37.184389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164780    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164783    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164786    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164789    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164791    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164794    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164797    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164801    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164805    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164808    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164811    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164816    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164819    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164822    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164825    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164828    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164831    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164834    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164837    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164840    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:37.185418 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164843    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164845    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164848    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164851    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164855    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164860    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164863    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164867    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164870    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164873    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164876    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164880    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164882    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164887    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164889    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164892    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164895    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164898    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164901    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:37.189218 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164904    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.164907    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165345    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165351    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165355    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165358    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165361    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165364    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165367    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165372    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165377    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165383    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165389    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165394    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165398    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165404    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165407    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165410    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165415    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:37.190610 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165420    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165424    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165427    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165430    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165432    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165436    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165439    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165442    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165445    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165448    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165451    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165454    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165457    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165460    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165462    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165465    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165468    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165470    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165473    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165476    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:37.193962 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165478    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165481    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165485    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165488    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165490    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165493    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165496    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165499    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165502    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165504    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165507    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165510    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165514    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165516    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165519    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165522    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165524    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165527    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165530    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165533    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:37.194986 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165550    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165555    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165558    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165561    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165563    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165566    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165569    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165572    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165575    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165578    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165581    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165584    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165587    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165590    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165595    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165598    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165601    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165604    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165606    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165609    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:37.199660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165612    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165614    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165617    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165620    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165625    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165629    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165631    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165634    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165637    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165639    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165642    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165645    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165648    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165651    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165653    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165656    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165659    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165662    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165664    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165667    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:37.200689 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165670    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.165673    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165752    2571 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165760    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165769    2571 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165775    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165779    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165783    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165787    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165792    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165795    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165798    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165802    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165806    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165810    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165813    2571 flags.go:64] FLAG: --cgroup-root=""
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165818    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165821    2571 flags.go:64] FLAG: --client-ca-file=""
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165825    2571 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165828    2571 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165834    2571 flags.go:64] FLAG: --cluster-domain=""
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165838    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165843    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165848    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 09:18:37.307679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165854    2571 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 09:18:37.235369 ip-10-0-138-208 systemd[1]: Started Kubernetes Kubelet.
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165858    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165862    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165865    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165869    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165872    2571 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165875    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165878    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165881    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165885    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165890    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165893    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165898    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165901    2571 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165904    2571 flags.go:64] FLAG: --enable-server="true"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165907    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165912    2571 flags.go:64] FLAG: --event-burst="100"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165916    2571 flags.go:64] FLAG: --event-qps="50"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165919    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165922    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165925    2571 flags.go:64] FLAG: --eviction-hard=""
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165929    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165932    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165935    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165940    2571 flags.go:64] FLAG: --eviction-soft=""
Jul 17 09:18:37.309631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165943    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165946    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165951    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165955    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165958    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165961    2571 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165964    2571 flags.go:64] FLAG: --feature-gates=""
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165968    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165972    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165975    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165978    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165981    2571 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165985    2571 flags.go:64] FLAG: --help="false"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165988    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-138-208.ec2.internal"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165992    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165995    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.165998    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166002    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166006    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166010    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166013    2571 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166016    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166019    2571 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 09:18:37.313753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166022    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166025    2571 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166028    2571 flags.go:64] FLAG: --kube-reserved=""
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166031    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166034    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166037    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166040    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166043    2571 flags.go:64] FLAG: --lock-file=""
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166048    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166051    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166055    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166060    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166065    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166068    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166071    2571 flags.go:64] FLAG: --logging-format="text"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166074    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166077    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166080    2571 flags.go:64] FLAG: --manifest-url=""
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166084    2571 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166089    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166092    2571 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166097    2571 flags.go:64] FLAG: --max-pods="110"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166100    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166103    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166106    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 09:18:37.315220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166109    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166112    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166115    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166120    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166128    2571 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166131    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166135    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166138    2571 flags.go:64] FLAG: --pod-cidr=""
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166141    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166146    2571 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166149    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166153    2571 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166156    2571 flags.go:64] FLAG: --port="10250"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166159    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166162    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cf677b3e3fbd1099"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166167    2571 flags.go:64] FLAG: --qos-reserved=""
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166170    2571 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166173    2571 flags.go:64] FLAG: --register-node="true"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166176    2571 flags.go:64] FLAG: --register-with-taints=""
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166180    2571 flags.go:64] FLAG: --registry-burst="10"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166184    2571 flags.go:64] FLAG: --registry-qps="5"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166187    2571 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166190    2571 flags.go:64] FLAG: --reserved-memory=""
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166194    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166197    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 09:18:37.318932 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166201    2571 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166204    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166207    2571 flags.go:64] FLAG: --runonce="false"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166210    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166214    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166217    2571 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166220    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166223    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166226    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166229    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166232    2571 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166237    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166240    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166243    2571 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166246    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166249    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166253    2571 flags.go:64] FLAG: --system-cgroups=""
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166256    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166261    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166265    2571 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166270    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166276    2571 flags.go:64] FLAG: --tls-min-version=""
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166283    2571 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166287    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166290    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 09:18:37.320392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166293    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166297    2571 flags.go:64] FLAG: --v="2"
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166301    2571 flags.go:64] FLAG: --version="false"
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166307    2571 flags.go:64] FLAG: --vmodule=""
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166311    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.166315    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166443    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166447    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166451    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166454    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166458    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166461    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166464    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166467    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166470    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166473    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166476    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166478    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166483    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166485    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166490    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:37.325396 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166494    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166498    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166503    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166506    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166509    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166512    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166515    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166517    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166522    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166525    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166528    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166530    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166533    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166549    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166552    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166556    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166559    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166562    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166565    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:37.326421 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166568    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166571    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166573    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166576    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166579    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166582    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166585    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166587    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166590    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166593    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166598    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166601    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166605    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166607    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166610    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166613    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166616    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166619    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166621    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166624    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:37.331966 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166627    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166632    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166634    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166639    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166642    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166646    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166649    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166652    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166655    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166659    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166662    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166665    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166668    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166670    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166673    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166676    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166679    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166681    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166684    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:37.333030 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166687    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166689    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166692    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166697    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166700    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166703    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166705    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166708    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166711    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166714    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166717    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166720    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166723    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166727    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166733    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166737    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166741    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166744    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166747    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166750    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:37.333795 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166754    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166757    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166760    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166765    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166768    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.166770    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.167610    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.175759    2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.175779    2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.175796    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.175807    2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175889    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175898    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175903    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:37.338127 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175907    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175912    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175915    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175919    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175923    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175926    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175930    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175934    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175937    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175941    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175947    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175952    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175956    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175960    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175963    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175967    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175971    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175975    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175979    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:37.338890 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175982    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175986    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175990    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175994    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.175997    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176001    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176007    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176011    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176015    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176018    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176022    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176025    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176029    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176032    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176036    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176039    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176042    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176046    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176049    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176053    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:37.339684 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176057    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176061    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176064    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176068    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176071    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176075    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176078    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176082    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176085    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176089    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176092    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176096    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176099    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176103    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176106    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176110    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176114    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176117    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176121    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176125    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:37.342437 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176128    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176132    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176135    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176139    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176142    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176146    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176150    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176153    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176157    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176160    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176164    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176167    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176171    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176174    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176178    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176181    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176184    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176188    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176191    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176195    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:37.343215 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176198    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176201    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176205    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176208    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176212    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176215    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176218    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176222    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176226    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176230    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176234    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176238    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176241    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176245    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176248    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176252    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:37.433976 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176255    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.176262    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176421    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176427    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176431    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176436    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176439    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176443    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176446    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176450    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176453    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176456    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176462    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176466    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176470    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:18:37.434894 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176474    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176478    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176482    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176485    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176490    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176494    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176498    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176502    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176505    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176509    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176513    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176516    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176520    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176524    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176527    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176531    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176534    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176552    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176556    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:18:37.437389 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176559    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176563    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176566    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176569    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176573    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176577    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176580    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176584    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176588    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176591    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176595    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176598    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176602    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176606    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176609    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176613    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176616    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176620    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176624    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176628    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:18:37.438092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176631    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176635    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176639    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176642    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176646    2571 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176650    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176654    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176659    2571 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176662    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176665    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176669    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176672    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176676    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176679    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176683    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176687    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176690    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176693    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176697    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176700    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:18:37.529167 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176703    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176707    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176710    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176714    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176717    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176721    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176726    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176731    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176735    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176738    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176742    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176745    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176749    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176752    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176756    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176760    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176763    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176768    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176771    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:18:37.530737 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176775    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176778    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176782    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176786    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176789    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176793    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176796    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:37.176799    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.176806    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.176941    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.179385    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.180907    2571 server.go:1019] "Starting client certificate rotation"
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.181018    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.181050    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 09:18:37.531622 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.183619    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 09:18:37.533428 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.188758    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 09:18:37.533428 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.199144    2571 log.go:25] "Validated CRI v1 runtime API"
Jul 17 09:18:37.533428 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.204373    2571 log.go:25] "Validated CRI v1 image API"
Jul 17 09:18:37.533428 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.205282    2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 09:18:37.533428 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.208607    2571 fs.go:135] Filesystem UUIDs: map[50022d18-288e-4345-b0f9-91985d331321:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b95aa112-4b5f-49c5-9c94-44cb1c2a2756:/dev/nvme0n1p3]
Jul 17 09:18:37.533428 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.208631    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.213092    2571 manager.go:216] Machine: {Timestamp:2026-07-17 09:18:37.212588634 +0000 UTC m=+1.277409350 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100120 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec288fb3eef9bb49d607c6aa0899a034 SystemUUID:ec288fb3-eef9-bb49-d607-c6aa0899a034 BootID:b711149e-819a-4396-b53d-78c2136efddf Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7a:24:3b:63:a9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7a:24:3b:63:a9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:44:60:66:f8:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.213201    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.213280    2571 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.216077    2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.216100    2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-208.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.217649    2571 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.217659    2571 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.217667    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.218449    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.219768    2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.223022    2571 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.223149    2571 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.223724    2571 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.223735    2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.223747    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.223757    2571 kubelet.go:398] "Adding apiserver pod source"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.223768    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 09:18:37.534580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.224185    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.224201    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.226602    2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.228805    2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.228833    2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.229919    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.229936    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.229945    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.229950    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.229962    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.229968    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.229976    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.229982    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.229989    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.230000    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.230016    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.230023    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.230045    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.230053    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.234149    2571 server.go:1268] "Started kubelet"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.234559    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.234843    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.234935    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.237108    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-208.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.237133    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-208.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.237244    2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.237351    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.238353    2571 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.243934    2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.244372    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.244385    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.244594    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.244902    2571 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.244930    2571 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.244902    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.245042    2571 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.245051    2571 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.245629    2571 factory.go:55] Registering systemd factory
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.245645    2571 factory.go:223] Registration of the systemd container factory successfully
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.245877    2571 factory.go:153] Registering CRI-O factory
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.245894    2571 factory.go:223] Registration of the crio container factory successfully
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.245951    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.245978    2571 factory.go:103] Registering Raw factory
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.245996    2571 manager.go:1196] Started watching for new ooms in manager
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.246426    2571 manager.go:349] Starting recovery of all containers
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.257668    2571 manager.go:354] Recovery completed
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.262115    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.263981    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:37.535750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.264027    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.264040    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.264602    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.264616    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.264614    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.264669    2571 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.265291    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.265316    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.265331    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.265929    2571 policy_none.go:49] "None policy: Start"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.265943    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.265955    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.267344    2571 policy_none.go:47] "Start"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.345425    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.435750    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.443893    2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-208.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.444283    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.441257    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-208.ec2.internal.18c309219698e90e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-208.ec2.internal,UID:ip-10-0-138-208.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-208.ec2.internal,},FirstTimestamp:2026-07-17 09:18:37.23402267 +0000 UTC m=+1.298843388,LastTimestamp:2026-07-17 09:18:37.23402267 +0000 UTC m=+1.298843388,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-208.ec2.internal,}"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.445704    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.457005    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-208.ec2.internal.18c3092198627a82  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-208.ec2.internal,UID:ip-10-0-138-208.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-208.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-208.ec2.internal,},FirstTimestamp:2026-07-17 09:18:37.264009858 +0000 UTC m=+1.328830582,LastTimestamp:2026-07-17 09:18:37.264009858 +0000 UTC m=+1.328830582,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-208.ec2.internal,}"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.459065    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hrk4j"
Jul 17 09:18:37.538064 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.466584    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-208.ec2.internal.18c309219862d398  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-208.ec2.internal,UID:ip-10-0-138-208.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-208.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-208.ec2.internal,},FirstTimestamp:2026-07-17 09:18:37.264032664 +0000 UTC m=+1.328853381,LastTimestamp:2026-07-17 09:18:37.264032664 +0000 UTC m=+1.328853381,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-208.ec2.internal,}"
Jul 17 09:18:37.540399 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.475936    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-208.ec2.internal.18c3092198630337  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-208.ec2.internal,UID:ip-10-0-138-208.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-208.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-208.ec2.internal,},FirstTimestamp:2026-07-17 09:18:37.264044855 +0000 UTC m=+1.328865577,LastTimestamp:2026-07-17 09:18:37.264044855 +0000 UTC m=+1.328865577,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-208.ec2.internal,}"
Jul 17 09:18:37.540399 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.477597    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hrk4j"
Jul 17 09:18:37.546516 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.546487    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:37.612508 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.612314    2571 manager.go:337] "Starting Device Plugin manager"
Jul 17 09:18:37.612508 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.612489    2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 09:18:37.612508 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.612500    2571 server.go:87] "Starting device plugin registration server"
Jul 17 09:18:37.612766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.612752    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 09:18:37.612830 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.612768    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 09:18:37.613195 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.613176    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 09:18:37.613305 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.613245    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 09:18:37.613305 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.613251    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 09:18:37.613966 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.613946    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 09:18:37.614055 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.614004    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:37.649722 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.649695    2571 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-208.ec2.internal\" not found" node="ip-10-0-138-208.ec2.internal"
Jul 17 09:18:37.713869 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.713838    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:37.715208 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.715189    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:37.715289 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.715230    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:37.715289 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.715245    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:37.715289 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.715278    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-208.ec2.internal"
Jul 17 09:18:37.725668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.725650    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-208.ec2.internal"
Jul 17 09:18:37.725738 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.725677    2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-208.ec2.internal\": node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:37.744175 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.744152    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:37.750303 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.750261    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 09:18:37.751489 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.751468    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 09:18:37.751615 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.751507    2571 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 09:18:37.751615 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.751530    2571 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 09:18:37.751615 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.751583    2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 09:18:37.753906 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.753888    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:18:37.850095 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.850056    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:37.852346 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.852302    2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-208.ec2.internal"]
Jul 17 09:18:37.852467 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.852384    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:37.853404 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.853387    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:37.853481 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.853423    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:37.853481 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.853435    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:37.854751 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.854737    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:37.854880 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.854865    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:37.854920 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.854912    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:37.855518 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.855502    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:37.855593 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.855532    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:37.855593 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.855556    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:37.855593 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.855538    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:37.855593 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.855584    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:37.855713 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.855596    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:37.856880 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.856864    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:37.856935 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.856892    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:18:37.858182 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.858147    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:18:37.858259 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.858214    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:18:37.858259 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.858233    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:18:37.880285 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.880218    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-208.ec2.internal\" not found" node="ip-10-0-138-208.ec2.internal"
Jul 17 09:18:37.884867 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.884849    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-208.ec2.internal\" not found" node="ip-10-0-138-208.ec2.internal"
Jul 17 09:18:37.950304 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:37.950269    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:37.950505 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.950337    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a06a1368bcefc63cd80f37a727f096c2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal\" (UID: \"a06a1368bcefc63cd80f37a727f096c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:37.950505 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.950361    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a06a1368bcefc63cd80f37a727f096c2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal\" (UID: \"a06a1368bcefc63cd80f37a727f096c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:37.950505 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:37.950379    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b1ab77a29ff5b283c6ddae565b57e84f-config\") pod \"kube-apiserver-proxy-ip-10-0-138-208.ec2.internal\" (UID: \"b1ab77a29ff5b283c6ddae565b57e84f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:38.050825 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:38.050789    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:38.050923 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.050885    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a06a1368bcefc63cd80f37a727f096c2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal\" (UID: \"a06a1368bcefc63cd80f37a727f096c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:38.050923 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.050911    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a06a1368bcefc63cd80f37a727f096c2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal\" (UID: \"a06a1368bcefc63cd80f37a727f096c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:38.051021 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.050936    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b1ab77a29ff5b283c6ddae565b57e84f-config\") pod \"kube-apiserver-proxy-ip-10-0-138-208.ec2.internal\" (UID: \"b1ab77a29ff5b283c6ddae565b57e84f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:38.051021 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.050985    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b1ab77a29ff5b283c6ddae565b57e84f-config\") pod \"kube-apiserver-proxy-ip-10-0-138-208.ec2.internal\" (UID: \"b1ab77a29ff5b283c6ddae565b57e84f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:38.051021 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.051002    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a06a1368bcefc63cd80f37a727f096c2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal\" (UID: \"a06a1368bcefc63cd80f37a727f096c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:38.051172 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.051019    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a06a1368bcefc63cd80f37a727f096c2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal\" (UID: \"a06a1368bcefc63cd80f37a727f096c2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:38.151327 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:38.151246    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:38.181719 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.181683    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 09:18:38.182128 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.181888    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 09:18:38.184033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.184018    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:38.187988 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.187716    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:38.251792 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:38.251759    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-208.ec2.internal\" not found"
Jul 17 09:18:38.277928 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.277897    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:18:38.345300 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.345269    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:38.361046 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.361022    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 09:18:38.363310 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.363296    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-208.ec2.internal"
Jul 17 09:18:38.376103 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.376084    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 09:18:38.437447 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.437394    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:18:38.472078 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.472051    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:18:38.481878 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.481835    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 09:13:37 +0000 UTC" deadline="2028-02-02 04:38:07.193453847 +0000 UTC"
Jul 17 09:18:38.481878 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.481874    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13555h19m28.711584158s"
Jul 17 09:18:38.871270 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:38.871237    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda06a1368bcefc63cd80f37a727f096c2.slice/crio-c768b1ae587c748e56b99e7df89e252b6c4ff8583e73dc0e6cf46ddc49656a9b WatchSource:0}: Error finding container c768b1ae587c748e56b99e7df89e252b6c4ff8583e73dc0e6cf46ddc49656a9b: Status 404 returned error can't find the container with id c768b1ae587c748e56b99e7df89e252b6c4ff8583e73dc0e6cf46ddc49656a9b
Jul 17 09:18:38.871746 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:38.871724    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ab77a29ff5b283c6ddae565b57e84f.slice/crio-435adc06781992e60dd9a631ce173a6ce9613f2d9b516fec0467da8b4638ec71 WatchSource:0}: Error finding container 435adc06781992e60dd9a631ce173a6ce9613f2d9b516fec0467da8b4638ec71: Status 404 returned error can't find the container with id 435adc06781992e60dd9a631ce173a6ce9613f2d9b516fec0467da8b4638ec71
Jul 17 09:18:38.877003 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:38.876984    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:18:39.224744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.224663    2571 apiserver.go:52] "Watching apiserver"
Jul 17 09:18:39.234962 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.234937    2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 09:18:39.235320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.235296    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-x524x","openshift-network-diagnostics/network-check-target-jlhnc","openshift-ovn-kubernetes/ovnkube-node-jhvwq","kube-system/global-pull-secret-syncer-5tsrt","kube-system/konnectivity-agent-hhg49","openshift-cluster-node-tuning-operator/tuned-64jxs","openshift-image-registry/node-ca-ksfkn","openshift-multus/network-metrics-daemon-sttnl","openshift-network-operator/iptables-alerter-c54cf","kube-system/kube-apiserver-proxy-ip-10-0-138-208.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz","openshift-dns/node-resolver-n5hmq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal","openshift-multus/multus-5zt6k"]
Jul 17 09:18:39.237531 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.237507    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.239519 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.239500    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:39.239605 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.239564    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:18:39.240761 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.240744    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.241873 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.241851    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:39.241955 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.241920    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:18:39.243164 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.243143    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:18:39.244388 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.244369    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.244757 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.244627    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 09:18:39.245833 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.245811    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ksfkn"
Jul 17 09:18:39.246596 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.246579    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 09:18:39.247016 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.247002    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:39.247088 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.247069    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:18:39.248422 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.248403    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c54cf"
Jul 17 09:18:39.249826 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.249807    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.251167 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.251149    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n5hmq"
Jul 17 09:18:39.252278 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.252260    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.256811    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7237e04-1a64-475d-bf54-112f171705fe-cni-binary-copy\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.256861    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd9h7\" (UniqueName: \"kubernetes.io/projected/e7237e04-1a64-475d-bf54-112f171705fe-kube-api-access-wd9h7\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.256896    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0970881d-9524-4242-bc47-7f0813bf2df1-dbus\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.256924    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.256957    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-lib-modules\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.256987    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-host\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257016    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-var-lib-openvswitch\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257046    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-cnibin\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257079    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e7237e04-1a64-475d-bf54-112f171705fe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257103    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-modprobe-d\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257132    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-sysconfig\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257179    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-systemd-units\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257210    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-run-openvswitch\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257235    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257557    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257590    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257804    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257851    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 09:18:39.259207 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.257240    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr897\" (UniqueName: \"kubernetes.io/projected/29ba670a-9f03-4942-be1e-79807bf25031-kube-api-access-jr897\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258059    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c0e214a-055e-4903-864b-7a86e3cc05c5-agent-certs\") pod \"konnectivity-agent-hhg49\" (UID: \"5c0e214a-055e-4903-864b-7a86e3cc05c5\") " pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258096    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-kubernetes\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258128    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-systemd\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258159    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6777b058-1852-426e-8fe2-83f71ed62122-etc-tuned\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258212    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6777b058-1852-426e-8fe2-83f71ed62122-tmp\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258243    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-run-netns\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258276    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-system-cni-dir\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258302    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-sysctl-d\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258332    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-run\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258352    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258360    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/53816944-d4bc-4e11-8c58-b9427233583a-host\") pod \"node-ca-ksfkn\" (UID: \"53816944-d4bc-4e11-8c58-b9427233583a\") " pod="openshift-image-registry/node-ca-ksfkn"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258395    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-run-systemd\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258406    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258426    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-node-log\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258453    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/29ba670a-9f03-4942-be1e-79807bf25031-env-overrides\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258486    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxzc4\" (UniqueName: \"kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4\") pod \"network-check-target-jlhnc\" (UID: \"0d2078c6-e7d7-495a-aa8f-45d4356cee4e\") " pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258516    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c0e214a-055e-4903-864b-7a86e3cc05c5-konnectivity-ca\") pod \"konnectivity-agent-hhg49\" (UID: \"5c0e214a-055e-4903-864b-7a86e3cc05c5\") " pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:18:39.260127 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258572    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e7237e04-1a64-475d-bf54-112f171705fe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258617    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-kubelet\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258704    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-os-release\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258737    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258737    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-sysctl-conf\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258769    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hlbf\" (UniqueName: \"kubernetes.io/projected/6777b058-1852-426e-8fe2-83f71ed62122-kube-api-access-4hlbf\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258788    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258819    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-etc-openvswitch\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258849    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-log-socket\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258875    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-cni-bin\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258907    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258940    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/29ba670a-9f03-4942-be1e-79807bf25031-ovnkube-config\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.258969    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259049    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259182    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b5s7r\""
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259225    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259186    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-sys\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259355    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-var-lib-kubelet\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259390    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/53816944-d4bc-4e11-8c58-b9427233583a-serviceca\") pod \"node-ca-ksfkn\" (UID: \"53816944-d4bc-4e11-8c58-b9427233583a\") " pod="openshift-image-registry/node-ca-ksfkn"
Jul 17 09:18:39.260758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259462    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5kqf\" (UniqueName: \"kubernetes.io/projected/53816944-d4bc-4e11-8c58-b9427233583a-kube-api-access-s5kqf\") pod \"node-ca-ksfkn\" (UID: \"53816944-d4bc-4e11-8c58-b9427233583a\") " pod="openshift-image-registry/node-ca-ksfkn"
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259512    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-slash\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259524    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259534    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-run-ovn\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259565    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0970881d-9524-4242-bc47-7f0813bf2df1-kubelet-config\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259597    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-run-ovn-kubernetes\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259629    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-cni-netd\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259683    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/29ba670a-9f03-4942-be1e-79807bf25031-ovn-node-metrics-cert\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259671    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q6q7c\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259711    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259723    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259725    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/29ba670a-9f03-4942-be1e-79807bf25031-ovnkube-script-lib\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259867    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.259901    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.260761    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.260768    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4sbq8\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.260765    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.260829    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-v5njt\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.260847    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.260765    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.260917    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h52bc\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.260985    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6mzm5\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.261004    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.261050    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.261056    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.261100    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.261143    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.261158    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mpnxw\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.261175    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-shd42\""
Jul 17 09:18:39.261678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.261191    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 09:18:39.262528 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.261253    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5qr6z\""
Jul 17 09:18:39.299977 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.299957    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 09:18:39.346269 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.346243    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 09:18:39.360396 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360364    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:39.360536 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360418    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/60829fa6-0828-485f-95cb-8184c374f75e-cni-binary-copy\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.360536 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360445    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-multus-conf-dir\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.360536 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360497    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv2vg\" (UniqueName: \"kubernetes.io/projected/60829fa6-0828-485f-95cb-8184c374f75e-kube-api-access-fv2vg\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.360688 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360543    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c0e214a-055e-4903-864b-7a86e3cc05c5-agent-certs\") pod \"konnectivity-agent-hhg49\" (UID: \"5c0e214a-055e-4903-864b-7a86e3cc05c5\") " pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:18:39.360688 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360614    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-systemd\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.360785 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360695    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-systemd\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.360785 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360699    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6777b058-1852-426e-8fe2-83f71ed62122-etc-tuned\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.360785 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360752    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6777b058-1852-426e-8fe2-83f71ed62122-tmp\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.360934 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360796    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-run-netns\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.360934 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360823    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-sysctl-d\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.360934 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360846    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/53816944-d4bc-4e11-8c58-b9427233583a-host\") pod \"node-ca-ksfkn\" (UID: \"53816944-d4bc-4e11-8c58-b9427233583a\") " pod="openshift-image-registry/node-ca-ksfkn"
Jul 17 09:18:39.360934 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360887    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-node-log\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360935    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-sys-fs\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360963    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-var-lib-cni-multus\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.360991    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzc4\" (UniqueName: \"kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4\") pod \"network-check-target-jlhnc\" (UID: \"0d2078c6-e7d7-495a-aa8f-45d4356cee4e\") " pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361018    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e7237e04-1a64-475d-bf54-112f171705fe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361002    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361033    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-sysctl-d\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361035    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-node-log\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361047    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-kubelet\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361083    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-run-netns\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361086    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-cnibin\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361117    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-kubelet\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361146    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/60829fa6-0828-485f-95cb-8184c374f75e-multus-daemon-config\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361172    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/53816944-d4bc-4e11-8c58-b9427233583a-host\") pod \"node-ca-ksfkn\" (UID: \"53816944-d4bc-4e11-8c58-b9427233583a\") " pod="openshift-image-registry/node-ca-ksfkn"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361186    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-run-multus-certs\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.361209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361215    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-os-release\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361242    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-sysctl-conf\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361267    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-log-socket\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361296    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361322    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-multus-cni-dir\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361324    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-os-release\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361347    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-os-release\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361356    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-log-socket\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361376    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-run-k8s-cni-cncf-io\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361394    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361405    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-var-lib-kubelet\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361443    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/53816944-d4bc-4e11-8c58-b9427233583a-serviceca\") pod \"node-ca-ksfkn\" (UID: \"53816944-d4bc-4e11-8c58-b9427233583a\") " pod="openshift-image-registry/node-ca-ksfkn"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361454    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-var-lib-kubelet\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361457    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-sysctl-conf\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361471    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-run-ovn-kubernetes\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361521    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-run-ovn-kubernetes\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361568    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-cni-netd\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.362033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361592    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-registration-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361615    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0970881d-9524-4242-bc47-7f0813bf2df1-kubelet-config\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361652    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-etc-selinux\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361674    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a14753f4-9bd4-4d68-8e03-8558b527cfec-iptables-alerter-script\") pod \"iptables-alerter-c54cf\" (UID: \"a14753f4-9bd4-4d68-8e03-8558b527cfec\") " pod="openshift-network-operator/iptables-alerter-c54cf"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361675    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-cni-netd\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361694    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/199a5035-60c1-4b0d-941e-4a1a20291037-tmp-dir\") pod \"node-resolver-n5hmq\" (UID: \"199a5035-60c1-4b0d-941e-4a1a20291037\") " pod="openshift-dns/node-resolver-n5hmq"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361716    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7237e04-1a64-475d-bf54-112f171705fe-cni-binary-copy\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361738    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0970881d-9524-4242-bc47-7f0813bf2df1-kubelet-config\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361751    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361806    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-lib-modules\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361829    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-host\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361853    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-run-openvswitch\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361882    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr897\" (UniqueName: \"kubernetes.io/projected/29ba670a-9f03-4942-be1e-79807bf25031-kube-api-access-jr897\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361908    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e7237e04-1a64-475d-bf54-112f171705fe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361932    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-modprobe-d\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361955    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-systemd-units\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361970    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e7237e04-1a64-475d-bf54-112f171705fe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.362910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361988    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/53816944-d4bc-4e11-8c58-b9427233583a-serviceca\") pod \"node-ca-ksfkn\" (UID: \"53816944-d4bc-4e11-8c58-b9427233583a\") " pod="openshift-image-registry/node-ca-ksfkn"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.362062    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362113    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-lib-modules\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.362131    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret podName:0970881d-9524-4242-bc47-7f0813bf2df1 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:39.86211023 +0000 UTC m=+3.926930948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret") pod "global-pull-secret-syncer-5tsrt" (UID: "0970881d-9524-4242-bc47-7f0813bf2df1") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362167    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-host\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.361977    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/199a5035-60c1-4b0d-941e-4a1a20291037-hosts-file\") pod \"node-resolver-n5hmq\" (UID: \"199a5035-60c1-4b0d-941e-4a1a20291037\") " pod="openshift-dns/node-resolver-n5hmq"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362211    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-run-openvswitch\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362252    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e7237e04-1a64-475d-bf54-112f171705fe-cni-binary-copy\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362272    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-systemd-units\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362250    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-kubernetes\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362302    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-kubernetes\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362314    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-device-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362342    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-etc-kubernetes\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362366    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a14753f4-9bd4-4d68-8e03-8558b527cfec-host-slash\") pod \"iptables-alerter-c54cf\" (UID: \"a14753f4-9bd4-4d68-8e03-8558b527cfec\") " pod="openshift-network-operator/iptables-alerter-c54cf"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362392    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95tt7\" (UniqueName: \"kubernetes.io/projected/a14753f4-9bd4-4d68-8e03-8558b527cfec-kube-api-access-95tt7\") pod \"iptables-alerter-c54cf\" (UID: \"a14753f4-9bd4-4d68-8e03-8558b527cfec\") " pod="openshift-network-operator/iptables-alerter-c54cf"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362398    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-modprobe-d\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362424    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-system-cni-dir\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.363766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362451    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-run\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362471    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-system-cni-dir\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362478    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-run-systemd\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362502    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-multus-socket-dir-parent\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362530    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c0e214a-055e-4903-864b-7a86e3cc05c5-konnectivity-ca\") pod \"konnectivity-agent-hhg49\" (UID: \"5c0e214a-055e-4903-864b-7a86e3cc05c5\") " pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362577    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmjjd\" (UniqueName: \"kubernetes.io/projected/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-kube-api-access-tmjjd\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362595    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-run-systemd\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362602    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-run\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362606    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz5fg\" (UniqueName: \"kubernetes.io/projected/0f229870-274c-459a-8e97-c8328417069a-kube-api-access-kz5fg\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362624    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e7237e04-1a64-475d-bf54-112f171705fe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362639    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-run-netns\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362680    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-var-lib-cni-bin\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362744    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-var-lib-kubelet\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362771    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-hostroot\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362800    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/29ba670a-9f03-4942-be1e-79807bf25031-env-overrides\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362843    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hlbf\" (UniqueName: \"kubernetes.io/projected/6777b058-1852-426e-8fe2-83f71ed62122-kube-api-access-4hlbf\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362884    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-etc-openvswitch\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.364609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.362962    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-cni-bin\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363009    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/29ba670a-9f03-4942-be1e-79807bf25031-ovnkube-config\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363036    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lktw6\" (UniqueName: \"kubernetes.io/projected/199a5035-60c1-4b0d-941e-4a1a20291037-kube-api-access-lktw6\") pod \"node-resolver-n5hmq\" (UID: \"199a5035-60c1-4b0d-941e-4a1a20291037\") " pod="openshift-dns/node-resolver-n5hmq"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363083    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363122    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-sys\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363157    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c0e214a-055e-4903-864b-7a86e3cc05c5-konnectivity-ca\") pod \"konnectivity-agent-hhg49\" (UID: \"5c0e214a-055e-4903-864b-7a86e3cc05c5\") " pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363145    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5kqf\" (UniqueName: \"kubernetes.io/projected/53816944-d4bc-4e11-8c58-b9427233583a-kube-api-access-s5kqf\") pod \"node-ca-ksfkn\" (UID: \"53816944-d4bc-4e11-8c58-b9427233583a\") " pod="openshift-image-registry/node-ca-ksfkn"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363224    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-slash\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363251    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-run-ovn\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363280    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/29ba670a-9f03-4942-be1e-79807bf25031-ovn-node-metrics-cert\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363292    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/29ba670a-9f03-4942-be1e-79807bf25031-env-overrides\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363308    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/29ba670a-9f03-4942-be1e-79807bf25031-ovnkube-script-lib\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363337    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363356    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-run-ovn\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363366    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd9h7\" (UniqueName: \"kubernetes.io/projected/e7237e04-1a64-475d-bf54-112f171705fe-kube-api-access-wd9h7\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363391    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0970881d-9524-4242-bc47-7f0813bf2df1-dbus\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363397    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-slash\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363438    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-var-lib-openvswitch\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363464    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-socket-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363489    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-system-cni-dir\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363519    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-cnibin\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363570    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-sysconfig\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363698    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-sys\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363763    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6777b058-1852-426e-8fe2-83f71ed62122-etc-sysconfig\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363793    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363848    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-var-lib-openvswitch\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363877    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/29ba670a-9f03-4942-be1e-79807bf25031-ovnkube-config\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363912    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e7237e04-1a64-475d-bf54-112f171705fe-cnibin\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363955    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-host-cni-bin\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.364016    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29ba670a-9f03-4942-be1e-79807bf25031-etc-openvswitch\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.363882    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0970881d-9524-4242-bc47-7f0813bf2df1-dbus\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.364517    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/29ba670a-9f03-4942-be1e-79807bf25031-ovnkube-script-lib\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.365153    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6777b058-1852-426e-8fe2-83f71ed62122-tmp\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.365442    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6777b058-1852-426e-8fe2-83f71ed62122-etc-tuned\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.365838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.365486    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c0e214a-055e-4903-864b-7a86e3cc05c5-agent-certs\") pod \"konnectivity-agent-hhg49\" (UID: \"5c0e214a-055e-4903-864b-7a86e3cc05c5\") " pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:18:39.366457 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.366235    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/29ba670a-9f03-4942-be1e-79807bf25031-ovn-node-metrics-cert\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.369358 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.369336    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bclbr"
Jul 17 09:18:39.376754 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.376733    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:18:39.382002 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.381983    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:39.382112 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.382007    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:39.382112 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.382020    2571 projected.go:196] Error preparing data for projected volume kube-api-access-qxzc4 for pod openshift-network-diagnostics/network-check-target-jlhnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:39.382112 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.382098    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4 podName:0d2078c6-e7d7-495a-aa8f-45d4356cee4e nodeName:}" failed. No retries permitted until 2026-07-17 09:18:39.882066278 +0000 UTC m=+3.946886997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qxzc4" (UniqueName: "kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4") pod "network-check-target-jlhnc" (UID: "0d2078c6-e7d7-495a-aa8f-45d4356cee4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:39.384668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.384645    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd9h7\" (UniqueName: \"kubernetes.io/projected/e7237e04-1a64-475d-bf54-112f171705fe-kube-api-access-wd9h7\") pod \"multus-additional-cni-plugins-x524x\" (UID: \"e7237e04-1a64-475d-bf54-112f171705fe\") " pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.385146 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.385126    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5kqf\" (UniqueName: \"kubernetes.io/projected/53816944-d4bc-4e11-8c58-b9427233583a-kube-api-access-s5kqf\") pod \"node-ca-ksfkn\" (UID: \"53816944-d4bc-4e11-8c58-b9427233583a\") " pod="openshift-image-registry/node-ca-ksfkn"
Jul 17 09:18:39.385226 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.385172    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr897\" (UniqueName: \"kubernetes.io/projected/29ba670a-9f03-4942-be1e-79807bf25031-kube-api-access-jr897\") pod \"ovnkube-node-jhvwq\" (UID: \"29ba670a-9f03-4942-be1e-79807bf25031\") " pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.387676 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.387653    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hlbf\" (UniqueName: \"kubernetes.io/projected/6777b058-1852-426e-8fe2-83f71ed62122-kube-api-access-4hlbf\") pod \"tuned-64jxs\" (UID: \"6777b058-1852-426e-8fe2-83f71ed62122\") " pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.389238 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.389215    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bclbr"
Jul 17 09:18:39.463978 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.463941    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-sys-fs\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.464149 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.463991    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-var-lib-cni-multus\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464149 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464033    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-cnibin\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464149 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464057    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/60829fa6-0828-485f-95cb-8184c374f75e-multus-daemon-config\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464149 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464065    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-sys-fs\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.464149 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464083    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-run-multus-certs\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464149 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464109    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-multus-cni-dir\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464149 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464132    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-cnibin\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464135    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-os-release\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464182    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-run-k8s-cni-cncf-io\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464198    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-multus-cni-dir\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464213    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-registration-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464240    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-etc-selinux\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464258    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-os-release\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464266    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a14753f4-9bd4-4d68-8e03-8558b527cfec-iptables-alerter-script\") pod \"iptables-alerter-c54cf\" (UID: \"a14753f4-9bd4-4d68-8e03-8558b527cfec\") " pod="openshift-network-operator/iptables-alerter-c54cf"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464292    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/199a5035-60c1-4b0d-941e-4a1a20291037-tmp-dir\") pod \"node-resolver-n5hmq\" (UID: \"199a5035-60c1-4b0d-941e-4a1a20291037\") " pod="openshift-dns/node-resolver-n5hmq"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464307    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-run-k8s-cni-cncf-io\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464340    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/199a5035-60c1-4b0d-941e-4a1a20291037-hosts-file\") pod \"node-resolver-n5hmq\" (UID: \"199a5035-60c1-4b0d-941e-4a1a20291037\") " pod="openshift-dns/node-resolver-n5hmq"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464366    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-device-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464371    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-var-lib-cni-multus\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464398    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-etc-kubernetes\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464421    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/199a5035-60c1-4b0d-941e-4a1a20291037-hosts-file\") pod \"node-resolver-n5hmq\" (UID: \"199a5035-60c1-4b0d-941e-4a1a20291037\") " pod="openshift-dns/node-resolver-n5hmq"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464424    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a14753f4-9bd4-4d68-8e03-8558b527cfec-host-slash\") pod \"iptables-alerter-c54cf\" (UID: \"a14753f4-9bd4-4d68-8e03-8558b527cfec\") " pod="openshift-network-operator/iptables-alerter-c54cf"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464455    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a14753f4-9bd4-4d68-8e03-8558b527cfec-host-slash\") pod \"iptables-alerter-c54cf\" (UID: \"a14753f4-9bd4-4d68-8e03-8558b527cfec\") " pod="openshift-network-operator/iptables-alerter-c54cf"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464459    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95tt7\" (UniqueName: \"kubernetes.io/projected/a14753f4-9bd4-4d68-8e03-8558b527cfec-kube-api-access-95tt7\") pod \"iptables-alerter-c54cf\" (UID: \"a14753f4-9bd4-4d68-8e03-8558b527cfec\") " pod="openshift-network-operator/iptables-alerter-c54cf"
Jul 17 09:18:39.464516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464488    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-multus-socket-dir-parent\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464535    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmjjd\" (UniqueName: \"kubernetes.io/projected/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-kube-api-access-tmjjd\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464563    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kz5fg\" (UniqueName: \"kubernetes.io/projected/0f229870-274c-459a-8e97-c8328417069a-kube-api-access-kz5fg\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464588    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-run-netns\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464611    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-var-lib-cni-bin\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464636    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-var-lib-kubelet\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464659    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-hostroot\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464700    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lktw6\" (UniqueName: \"kubernetes.io/projected/199a5035-60c1-4b0d-941e-4a1a20291037-kube-api-access-lktw6\") pod \"node-resolver-n5hmq\" (UID: \"199a5035-60c1-4b0d-941e-4a1a20291037\") " pod="openshift-dns/node-resolver-n5hmq"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464735    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464763    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/199a5035-60c1-4b0d-941e-4a1a20291037-tmp-dir\") pod \"node-resolver-n5hmq\" (UID: \"199a5035-60c1-4b0d-941e-4a1a20291037\") " pod="openshift-dns/node-resolver-n5hmq"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464764    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-socket-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464802    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-system-cni-dir\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464830    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464857    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/60829fa6-0828-485f-95cb-8184c374f75e-cni-binary-copy\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464862    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-etc-kubernetes\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464891    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-multus-conf-dir\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464917    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv2vg\" (UniqueName: \"kubernetes.io/projected/60829fa6-0828-485f-95cb-8184c374f75e-kube-api-access-fv2vg\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.465392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464959    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a14753f4-9bd4-4d68-8e03-8558b527cfec-iptables-alerter-script\") pod \"iptables-alerter-c54cf\" (UID: \"a14753f4-9bd4-4d68-8e03-8558b527cfec\") " pod="openshift-network-operator/iptables-alerter-c54cf"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.465199    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-multus-socket-dir-parent\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.465458    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/60829fa6-0828-485f-95cb-8184c374f75e-multus-daemon-config\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.465527    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-registration-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.465598    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-run-netns\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.465650    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-var-lib-cni-bin\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.465651    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-etc-selinux\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.465705    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-hostroot\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.465711    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-device-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.465787    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-var-lib-kubelet\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.465846    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-system-cni-dir\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.465900    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.465991    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.466020    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0f229870-274c-459a-8e97-c8328417069a-socket-dir\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.466049    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs podName:ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea nodeName:}" failed. No retries permitted until 2026-07-17 09:18:39.96603005 +0000 UTC m=+4.030850766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs") pod "network-metrics-daemon-sttnl" (UID: "ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.464820    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-host-run-multus-certs\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.466301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.466068    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/60829fa6-0828-485f-95cb-8184c374f75e-multus-conf-dir\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.467061 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.466477    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/60829fa6-0828-485f-95cb-8184c374f75e-cni-binary-copy\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.493051 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.492985    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95tt7\" (UniqueName: \"kubernetes.io/projected/a14753f4-9bd4-4d68-8e03-8558b527cfec-kube-api-access-95tt7\") pod \"iptables-alerter-c54cf\" (UID: \"a14753f4-9bd4-4d68-8e03-8558b527cfec\") " pod="openshift-network-operator/iptables-alerter-c54cf"
Jul 17 09:18:39.493449 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.493426    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lktw6\" (UniqueName: \"kubernetes.io/projected/199a5035-60c1-4b0d-941e-4a1a20291037-kube-api-access-lktw6\") pod \"node-resolver-n5hmq\" (UID: \"199a5035-60c1-4b0d-941e-4a1a20291037\") " pod="openshift-dns/node-resolver-n5hmq"
Jul 17 09:18:39.493879 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.493858    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz5fg\" (UniqueName: \"kubernetes.io/projected/0f229870-274c-459a-8e97-c8328417069a-kube-api-access-kz5fg\") pod \"aws-ebs-csi-driver-node-jxmmz\" (UID: \"0f229870-274c-459a-8e97-c8328417069a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.499926 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.499898    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmjjd\" (UniqueName: \"kubernetes.io/projected/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-kube-api-access-tmjjd\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:39.500410 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.500380    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv2vg\" (UniqueName: \"kubernetes.io/projected/60829fa6-0828-485f-95cb-8184c374f75e-kube-api-access-fv2vg\") pod \"multus-5zt6k\" (UID: \"60829fa6-0828-485f-95cb-8184c374f75e\") " pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.547986 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.547954    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-x524x"
Jul 17 09:18:39.552883 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.552862    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:18:39.562778 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:39.562752    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7237e04_1a64_475d_bf54_112f171705fe.slice/crio-c9c0d931286ac9e90b80e5c7d2b6e80f706a8742f42094e96b18ba823be918ea WatchSource:0}: Error finding container c9c0d931286ac9e90b80e5c7d2b6e80f706a8742f42094e96b18ba823be918ea: Status 404 returned error can't find the container with id c9c0d931286ac9e90b80e5c7d2b6e80f706a8742f42094e96b18ba823be918ea
Jul 17 09:18:39.564157 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.564132    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:18:39.566160 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:39.566134    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ba670a_9f03_4942_be1e_79807bf25031.slice/crio-ab37a204c1b077d2ff742ba80cf566b322a66f9ab01337005b397490327d5ee5 WatchSource:0}: Error finding container ab37a204c1b077d2ff742ba80cf566b322a66f9ab01337005b397490327d5ee5: Status 404 returned error can't find the container with id ab37a204c1b077d2ff742ba80cf566b322a66f9ab01337005b397490327d5ee5
Jul 17 09:18:39.568117 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.567953    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-64jxs"
Jul 17 09:18:39.574835 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.574647    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ksfkn"
Jul 17 09:18:39.581483 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.581343    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c54cf"
Jul 17 09:18:39.584947 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:39.584922    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6777b058_1852_426e_8fe2_83f71ed62122.slice/crio-3f2cbc31f34b7a2b4dedbb7b45b44476ea27f6d62859cbfb4bf6936e27e39f5c WatchSource:0}: Error finding container 3f2cbc31f34b7a2b4dedbb7b45b44476ea27f6d62859cbfb4bf6936e27e39f5c: Status 404 returned error can't find the container with id 3f2cbc31f34b7a2b4dedbb7b45b44476ea27f6d62859cbfb4bf6936e27e39f5c
Jul 17 09:18:39.585555 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.585510    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz"
Jul 17 09:18:39.591291 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.591266    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n5hmq"
Jul 17 09:18:39.595889 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.595869    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5zt6k"
Jul 17 09:18:39.605785 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:39.605744    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f229870_274c_459a_8e97_c8328417069a.slice/crio-4b0b2f2f790a64de373043e6ef73dd80173e930d6838af0abb4b25c819e5e0c4 WatchSource:0}: Error finding container 4b0b2f2f790a64de373043e6ef73dd80173e930d6838af0abb4b25c819e5e0c4: Status 404 returned error can't find the container with id 4b0b2f2f790a64de373043e6ef73dd80173e930d6838af0abb4b25c819e5e0c4
Jul 17 09:18:39.609388 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:18:39.609360    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda14753f4_9bd4_4d68_8e03_8558b527cfec.slice/crio-c89db74c246c75c8cfec8caf89b60f4bb5ff18d91237370508cc5a780bd186ca WatchSource:0}: Error finding container c89db74c246c75c8cfec8caf89b60f4bb5ff18d91237370508cc5a780bd186ca: Status 404 returned error can't find the container with id c89db74c246c75c8cfec8caf89b60f4bb5ff18d91237370508cc5a780bd186ca
Jul 17 09:18:39.758719 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.758603    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal" event={"ID":"a06a1368bcefc63cd80f37a727f096c2","Type":"ContainerStarted","Data":"c768b1ae587c748e56b99e7df89e252b6c4ff8583e73dc0e6cf46ddc49656a9b"}
Jul 17 09:18:39.759877 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.759849    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n5hmq" event={"ID":"199a5035-60c1-4b0d-941e-4a1a20291037","Type":"ContainerStarted","Data":"f7681fc09f649fa07f2e9390e528f74204e868c40a1a3c058982a1c5106e2cf8"}
Jul 17 09:18:39.761979 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.761955    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ksfkn" event={"ID":"53816944-d4bc-4e11-8c58-b9427233583a","Type":"ContainerStarted","Data":"1acdf328b6ba81289c0ab609fc46f9b98be15dda236e493c58a10f6aec0dceea"}
Jul 17 09:18:39.763702 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.763651    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hhg49" event={"ID":"5c0e214a-055e-4903-864b-7a86e3cc05c5","Type":"ContainerStarted","Data":"9c06f7a58cb0ea9a12a3f69e4c6bbb1bae3aa7fda38997a2edc801be5844eb0f"}
Jul 17 09:18:39.765239 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.765193    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-208.ec2.internal" event={"ID":"b1ab77a29ff5b283c6ddae565b57e84f","Type":"ContainerStarted","Data":"435adc06781992e60dd9a631ce173a6ce9613f2d9b516fec0467da8b4638ec71"}
Jul 17 09:18:39.766755 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.766722    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5zt6k" event={"ID":"60829fa6-0828-485f-95cb-8184c374f75e","Type":"ContainerStarted","Data":"552febc9099b840948e0d3d3746bf1ca556f9bf29750a42a36bf9b434d6b0235"}
Jul 17 09:18:39.768221 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.768190    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz" event={"ID":"0f229870-274c-459a-8e97-c8328417069a","Type":"ContainerStarted","Data":"4b0b2f2f790a64de373043e6ef73dd80173e930d6838af0abb4b25c819e5e0c4"}
Jul 17 09:18:39.769645 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.769618    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c54cf" event={"ID":"a14753f4-9bd4-4d68-8e03-8558b527cfec","Type":"ContainerStarted","Data":"c89db74c246c75c8cfec8caf89b60f4bb5ff18d91237370508cc5a780bd186ca"}
Jul 17 09:18:39.771187 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.771162    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-64jxs" event={"ID":"6777b058-1852-426e-8fe2-83f71ed62122","Type":"ContainerStarted","Data":"3f2cbc31f34b7a2b4dedbb7b45b44476ea27f6d62859cbfb4bf6936e27e39f5c"}
Jul 17 09:18:39.772573 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.772551    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq" event={"ID":"29ba670a-9f03-4942-be1e-79807bf25031","Type":"ContainerStarted","Data":"ab37a204c1b077d2ff742ba80cf566b322a66f9ab01337005b397490327d5ee5"}
Jul 17 09:18:39.774557 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.774514    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x524x" event={"ID":"e7237e04-1a64-475d-bf54-112f171705fe","Type":"ContainerStarted","Data":"c9c0d931286ac9e90b80e5c7d2b6e80f706a8742f42094e96b18ba823be918ea"}
Jul 17 09:18:39.867209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.867172    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:39.867374 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.867346    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:39.867451 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.867434    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret podName:0970881d-9524-4242-bc47-7f0813bf2df1 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:40.867412036 +0000 UTC m=+4.932232755 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret") pod "global-pull-secret-syncer-5tsrt" (UID: "0970881d-9524-4242-bc47-7f0813bf2df1") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:39.968187 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.968144    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:39.968187 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:39.968204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzc4\" (UniqueName: \"kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4\") pod \"network-check-target-jlhnc\" (UID: \"0d2078c6-e7d7-495a-aa8f-45d4356cee4e\") " pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:39.968446 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.968333    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:39.968446 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.968371    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:39.968446 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.968402    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:39.968446 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.968416    2571 projected.go:196] Error preparing data for projected volume kube-api-access-qxzc4 for pod openshift-network-diagnostics/network-check-target-jlhnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:39.968446 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.968426    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs podName:ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea nodeName:}" failed. No retries permitted until 2026-07-17 09:18:40.968402337 +0000 UTC m=+5.033223038 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs") pod "network-metrics-daemon-sttnl" (UID: "ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:39.968744 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:39.968487    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4 podName:0d2078c6-e7d7-495a-aa8f-45d4356cee4e nodeName:}" failed. No retries permitted until 2026-07-17 09:18:40.968454944 +0000 UTC m=+5.033275645 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxzc4" (UniqueName: "kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4") pod "network-check-target-jlhnc" (UID: "0d2078c6-e7d7-495a-aa8f-45d4356cee4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:40.392035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:40.391963    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 09:13:39 +0000 UTC" deadline="2028-04-05 16:52:08.767281612 +0000 UTC"
Jul 17 09:18:40.392035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:40.392024    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15079h33m28.375261201s"
Jul 17 09:18:40.752070 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:40.752034    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:40.752247 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:40.752074    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:40.752247 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:40.752034    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:40.752247 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:40.752185    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:18:40.752433 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:40.752303    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:18:40.752433 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:40.752384    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:18:40.876717 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:40.876678    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:40.876918 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:40.876907    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:40.876997 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:40.876975    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret podName:0970881d-9524-4242-bc47-7f0813bf2df1 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:42.876955353 +0000 UTC m=+6.941776061 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret") pod "global-pull-secret-syncer-5tsrt" (UID: "0970881d-9524-4242-bc47-7f0813bf2df1") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:40.978606 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:40.977596    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:40.978606 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:40.977655    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzc4\" (UniqueName: \"kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4\") pod \"network-check-target-jlhnc\" (UID: \"0d2078c6-e7d7-495a-aa8f-45d4356cee4e\") " pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:40.978606 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:40.977842    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:40.978606 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:40.977861    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:40.978606 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:40.977874    2571 projected.go:196] Error preparing data for projected volume kube-api-access-qxzc4 for pod openshift-network-diagnostics/network-check-target-jlhnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:40.978606 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:40.977934    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4 podName:0d2078c6-e7d7-495a-aa8f-45d4356cee4e nodeName:}" failed. No retries permitted until 2026-07-17 09:18:42.977915649 +0000 UTC m=+7.042736355 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxzc4" (UniqueName: "kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4") pod "network-check-target-jlhnc" (UID: "0d2078c6-e7d7-495a-aa8f-45d4356cee4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:40.978606 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:40.978449    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:40.978606 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:40.978508    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs podName:ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea nodeName:}" failed. No retries permitted until 2026-07-17 09:18:42.978489593 +0000 UTC m=+7.043310299 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs") pod "network-metrics-daemon-sttnl" (UID: "ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:41.392851 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:41.392729    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 09:13:39 +0000 UTC" deadline="2028-01-20 20:16:42.316005643 +0000 UTC"
Jul 17 09:18:41.392851 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:41.392771    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13258h58m0.923239122s"
Jul 17 09:18:41.786843 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:41.786480    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-208.ec2.internal" event={"ID":"b1ab77a29ff5b283c6ddae565b57e84f","Type":"ContainerStarted","Data":"23b9b838efab61b8ea37518c0c0c2acfbf6a56e23a27662039894de69d35c5f4"}
Jul 17 09:18:42.751982 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:42.751942    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:42.752460 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:42.752087    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:18:42.752580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:42.752500    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:42.752652 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:42.752631    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:18:42.752709 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:42.752692    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:42.752861 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:42.752764    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:18:42.795035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:42.794453    2571 generic.go:356] "Generic (PLEG): container finished" podID="a06a1368bcefc63cd80f37a727f096c2" containerID="09ef38c616fafb36e6f2bccd296f5f9472e6271bbcc012733dd6eb4f079214c4" exitCode=0
Jul 17 09:18:42.795035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:42.794570    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal" event={"ID":"a06a1368bcefc63cd80f37a727f096c2","Type":"ContainerDied","Data":"09ef38c616fafb36e6f2bccd296f5f9472e6271bbcc012733dd6eb4f079214c4"}
Jul 17 09:18:42.817802 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:42.817741    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-208.ec2.internal" podStartSLOduration=4.817715303 podStartE2EDuration="4.817715303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:41.80384961 +0000 UTC m=+5.868670336" watchObservedRunningTime="2026-07-17 09:18:42.817715303 +0000 UTC m=+6.882536025"
Jul 17 09:18:42.895214 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:42.895139    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:42.895401 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:42.895306    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:42.895401 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:42.895359    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret podName:0970881d-9524-4242-bc47-7f0813bf2df1 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:46.895343966 +0000 UTC m=+10.960164668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret") pod "global-pull-secret-syncer-5tsrt" (UID: "0970881d-9524-4242-bc47-7f0813bf2df1") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:42.996679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:42.995940    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzc4\" (UniqueName: \"kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4\") pod \"network-check-target-jlhnc\" (UID: \"0d2078c6-e7d7-495a-aa8f-45d4356cee4e\") " pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:42.996679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:42.996033    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:42.996679 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:42.996147    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:42.996679 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:42.996202    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs podName:ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea nodeName:}" failed. No retries permitted until 2026-07-17 09:18:46.996185704 +0000 UTC m=+11.061006405 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs") pod "network-metrics-daemon-sttnl" (UID: "ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:42.996679 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:42.996574    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:42.996679 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:42.996589    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:42.996679 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:42.996602    2571 projected.go:196] Error preparing data for projected volume kube-api-access-qxzc4 for pod openshift-network-diagnostics/network-check-target-jlhnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:42.996679 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:42.996651    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4 podName:0d2078c6-e7d7-495a-aa8f-45d4356cee4e nodeName:}" failed. No retries permitted until 2026-07-17 09:18:46.996637851 +0000 UTC m=+11.061458551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxzc4" (UniqueName: "kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4") pod "network-check-target-jlhnc" (UID: "0d2078c6-e7d7-495a-aa8f-45d4356cee4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:44.751788 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:44.751757    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:44.752292 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:44.751794    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:44.752292 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:44.751794    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:44.752292 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:44.751862    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:18:44.752292 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:44.751978    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:18:44.752292 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:44.752083    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:18:46.752680 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:46.752333    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:46.752680 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:46.752382    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:46.752680 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:46.752341    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:46.752680 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:46.752487    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:18:46.752680 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:46.752630    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:18:46.753268 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:46.752704    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:18:46.931623 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:46.931553    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:46.931804 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:46.931705    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:46.931804 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:46.931791    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret podName:0970881d-9524-4242-bc47-7f0813bf2df1 nodeName:}" failed. No retries permitted until 2026-07-17 09:18:54.931768477 +0000 UTC m=+18.996589178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret") pod "global-pull-secret-syncer-5tsrt" (UID: "0970881d-9524-4242-bc47-7f0813bf2df1") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:47.032931 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:47.032846    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:47.032931 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:47.032909    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzc4\" (UniqueName: \"kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4\") pod \"network-check-target-jlhnc\" (UID: \"0d2078c6-e7d7-495a-aa8f-45d4356cee4e\") " pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:47.033129 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:47.033003    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:47.033129 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:47.033076    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs podName:ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea nodeName:}" failed. No retries permitted until 2026-07-17 09:18:55.03305798 +0000 UTC m=+19.097878683 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs") pod "network-metrics-daemon-sttnl" (UID: "ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:47.033237 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:47.033197    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:47.033237 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:47.033224    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:47.033237 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:47.033238    2571 projected.go:196] Error preparing data for projected volume kube-api-access-qxzc4 for pod openshift-network-diagnostics/network-check-target-jlhnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:47.033364 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:47.033300    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4 podName:0d2078c6-e7d7-495a-aa8f-45d4356cee4e nodeName:}" failed. No retries permitted until 2026-07-17 09:18:55.033283514 +0000 UTC m=+19.098104227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxzc4" (UniqueName: "kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4") pod "network-check-target-jlhnc" (UID: "0d2078c6-e7d7-495a-aa8f-45d4356cee4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:48.752675 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:48.752636    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:48.753152 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:48.752636    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:48.753152 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:48.752774    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:18:48.753152 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:48.752870    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:18:48.753152 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:48.752643    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:48.753152 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:48.752959    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:18:50.751800 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:50.751762    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:50.751800 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:50.751799    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:50.752581 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:50.751762    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:50.752581 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:50.751891    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:18:50.752581 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:50.751945    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:18:50.752581 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:50.752020    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:18:52.752518 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:52.752456    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:52.752975 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:52.752530    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:52.752975 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:52.752475    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:52.752975 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:52.752591    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:18:52.752975 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:52.752683    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:18:52.752975 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:52.752741    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:18:54.752372 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:54.752331    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:54.752372 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:54.752348    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:54.752372 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:54.752374    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:54.752951 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:54.752481    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:18:54.752951 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:54.752606    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:18:54.752951 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:54.752701    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:18:54.991647 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:54.991568    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:54.991923 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:54.991708    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:54.991923 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:54.991778    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret podName:0970881d-9524-4242-bc47-7f0813bf2df1 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:10.991758402 +0000 UTC m=+35.056579104 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret") pod "global-pull-secret-syncer-5tsrt" (UID: "0970881d-9524-4242-bc47-7f0813bf2df1") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:18:55.092348 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:55.092259    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:55.092348 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:55.092323    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzc4\" (UniqueName: \"kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4\") pod \"network-check-target-jlhnc\" (UID: \"0d2078c6-e7d7-495a-aa8f-45d4356cee4e\") " pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:55.092616 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:55.092410    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:55.092616 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:55.092452    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:18:55.092616 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:55.092489    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs podName:ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea nodeName:}" failed. No retries permitted until 2026-07-17 09:19:11.092458607 +0000 UTC m=+35.157279307 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs") pod "network-metrics-daemon-sttnl" (UID: "ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:18:55.092616 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:55.092490    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:18:55.092616 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:55.092511    2571 projected.go:196] Error preparing data for projected volume kube-api-access-qxzc4 for pod openshift-network-diagnostics/network-check-target-jlhnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:55.092616 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:55.092544    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4 podName:0d2078c6-e7d7-495a-aa8f-45d4356cee4e nodeName:}" failed. No retries permitted until 2026-07-17 09:19:11.092534893 +0000 UTC m=+35.157355594 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxzc4" (UniqueName: "kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4") pod "network-check-target-jlhnc" (UID: "0d2078c6-e7d7-495a-aa8f-45d4356cee4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:18:56.752163 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:56.752123    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:56.752611 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:56.752125    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:56.752611 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:56.752262    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:18:56.752611 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:56.752125    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:56.752611 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:56.752341    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:18:56.752611 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:56.752426    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:18:58.751768 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:58.751727    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:18:58.751768 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:58.751748    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:18:58.752269 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:58.751841    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:18:58.752269 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:58.751851    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:18:58.752269 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:58.751956    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:18:58.752269 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:18:58.752070    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:18:59.830576 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.830307    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5zt6k" event={"ID":"60829fa6-0828-485f-95cb-8184c374f75e","Type":"ContainerStarted","Data":"6e02b66ab84c8bb62fb43e962b5e9f1a5ec44e5c7812ac849a6c8d2f1bb01397"}
Jul 17 09:18:59.832326 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.832299    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz" event={"ID":"0f229870-274c-459a-8e97-c8328417069a","Type":"ContainerStarted","Data":"c93ea0ab37bbb41f46212c02ced679a71a1dff48dfa7b8658476a65fe3635734"}
Jul 17 09:18:59.833815 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.833793    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-64jxs" event={"ID":"6777b058-1852-426e-8fe2-83f71ed62122","Type":"ContainerStarted","Data":"10b50ddab6246f71f6bee6acafbb3df1f607d91165f819e3d3b64edf27acb04f"}
Jul 17 09:18:59.836173 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.836155    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:18:59.836453 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.836430    2571 generic.go:356] "Generic (PLEG): container finished" podID="29ba670a-9f03-4942-be1e-79807bf25031" containerID="87f89218da7744c17f5af1fe2c7bad7f9b0bba1b1487e085c4786d24a98f72e6" exitCode=1
Jul 17 09:18:59.836538 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.836483    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq" event={"ID":"29ba670a-9f03-4942-be1e-79807bf25031","Type":"ContainerStarted","Data":"fe850a557a50996c807e885bfd46285ff5c55e457c6185cc9462f5eb15e93b2a"}
Jul 17 09:18:59.836538 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.836536    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq" event={"ID":"29ba670a-9f03-4942-be1e-79807bf25031","Type":"ContainerStarted","Data":"0dac84e6c476be707eccab60c3907ceb9dc78728bea0b0d44da2487569051f34"}
Jul 17 09:18:59.836636 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.836566    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq" event={"ID":"29ba670a-9f03-4942-be1e-79807bf25031","Type":"ContainerStarted","Data":"ecf2d2555011bd9b6bf861fc4b5b333036ecf3745e0ff8f982b73920dfa77a2a"}
Jul 17 09:18:59.836636 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.836594    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq" event={"ID":"29ba670a-9f03-4942-be1e-79807bf25031","Type":"ContainerDied","Data":"87f89218da7744c17f5af1fe2c7bad7f9b0bba1b1487e085c4786d24a98f72e6"}
Jul 17 09:18:59.836712 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.836637    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq" event={"ID":"29ba670a-9f03-4942-be1e-79807bf25031","Type":"ContainerStarted","Data":"30b39e35b0e798c22279a43a4e9def20aefd2e17938fdac33b04165c73ba610c"}
Jul 17 09:18:59.837997 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.837975    2571 generic.go:356] "Generic (PLEG): container finished" podID="e7237e04-1a64-475d-bf54-112f171705fe" containerID="5e0b29fbbdce45c5c13b66736c3ded72f8d31fd815e386c89a41bd8a0bc17df8" exitCode=0
Jul 17 09:18:59.838104 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.838048    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x524x" event={"ID":"e7237e04-1a64-475d-bf54-112f171705fe","Type":"ContainerDied","Data":"5e0b29fbbdce45c5c13b66736c3ded72f8d31fd815e386c89a41bd8a0bc17df8"}
Jul 17 09:18:59.840078 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.840059    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal" event={"ID":"a06a1368bcefc63cd80f37a727f096c2","Type":"ContainerStarted","Data":"e240f0eac9e6c1953deb290700365f877beae34e094c0d6453f9db821519b598"}
Jul 17 09:18:59.841606 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.841580    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n5hmq" event={"ID":"199a5035-60c1-4b0d-941e-4a1a20291037","Type":"ContainerStarted","Data":"b7a0c30ab9421944d64d72902e5a07fce34e241c2b10e9487df14808eaf330df"}
Jul 17 09:18:59.842948 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.842926    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ksfkn" event={"ID":"53816944-d4bc-4e11-8c58-b9427233583a","Type":"ContainerStarted","Data":"de3fd4099bb313dcd9d1901c07ecce844d181bc8719a3e44b0420fc8991071cc"}
Jul 17 09:18:59.844416 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.844395    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hhg49" event={"ID":"5c0e214a-055e-4903-864b-7a86e3cc05c5","Type":"ContainerStarted","Data":"14ee72505f5a5c2862e9e143425a8a66c36e441222256e902693de7dfd84d3e7"}
Jul 17 09:18:59.854655 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.854617    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5zt6k" podStartSLOduration=1.900753189 podStartE2EDuration="21.854604695s" totalImagesPullingTime="19.953851506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:59.854078896 +0000 UTC m=+23.918899615" watchObservedRunningTime="2026-07-17 09:18:59.854604695 +0000 UTC m=+23.919425417"
Jul 17 09:18:59.873644 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.873602    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n5hmq" podStartSLOduration=2.225503922 podStartE2EDuration="21.873585613s" totalImagesPullingTime="19.648081691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:59.87331093 +0000 UTC m=+23.938131652" watchObservedRunningTime="2026-07-17 09:18:59.873585613 +0000 UTC m=+23.938406335"
Jul 17 09:18:59.894113 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.894070    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-208.ec2.internal" podStartSLOduration=21.894059318 podStartE2EDuration="21.894059318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:59.894054326 +0000 UTC m=+23.958875049" watchObservedRunningTime="2026-07-17 09:18:59.894059318 +0000 UTC m=+23.958880040"
Jul 17 09:18:59.952640 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.952582    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hhg49" podStartSLOduration=2.269713098 podStartE2EDuration="21.952561726s" totalImagesPullingTime="19.682848628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:59.95208678 +0000 UTC m=+24.016907502" watchObservedRunningTime="2026-07-17 09:18:59.952561726 +0000 UTC m=+24.017382449"
Jul 17 09:18:59.974417 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.974357    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-64jxs" podStartSLOduration=2.294756235 podStartE2EDuration="21.974339435s" totalImagesPullingTime="19.6795832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:59.97412655 +0000 UTC m=+24.038947271" watchObservedRunningTime="2026-07-17 09:18:59.974339435 +0000 UTC m=+24.039160166"
Jul 17 09:18:59.990564 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:18:59.990511    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ksfkn" podStartSLOduration=2.3208794360000002 podStartE2EDuration="21.990490851s" totalImagesPullingTime="19.669611415s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:18:59.990205701 +0000 UTC m=+24.055026423" watchObservedRunningTime="2026-07-17 09:18:59.990490851 +0000 UTC m=+24.055311576"
Jul 17 09:19:00.752684 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:00.752649    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:00.752684 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:00.752678    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:19:00.752875 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:00.752783    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:19:00.752975 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:00.752931    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:19:00.753099 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:00.753018    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:00.753164 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:00.753145    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:19:00.782443 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:00.782411    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 09:19:00.849025 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:00.848942    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz" event={"ID":"0f229870-274c-459a-8e97-c8328417069a","Type":"ContainerStarted","Data":"e3ed4a3dd71311285d57648e506f5d968376490334144c8c45d61447ca741a20"}
Jul 17 09:19:00.850357 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:00.850335    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c54cf" event={"ID":"a14753f4-9bd4-4d68-8e03-8558b527cfec","Type":"ContainerStarted","Data":"07e46a31782c32d60402a684628750336552326d5b996b8e37687203b54f57ec"}
Jul 17 09:19:00.853182 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:00.853162    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:19:00.853622 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:00.853597    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq" event={"ID":"29ba670a-9f03-4942-be1e-79807bf25031","Type":"ContainerStarted","Data":"12d87be183e26f2b3a394ba37582fa6a6f9191f23bdd13356241544dc37097b7"}
Jul 17 09:19:00.871581 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:00.871529    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c54cf" podStartSLOduration=3.216551346 podStartE2EDuration="22.871511944s" totalImagesPullingTime="19.654960598s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:00.870910068 +0000 UTC m=+24.935730791" watchObservedRunningTime="2026-07-17 09:19:00.871511944 +0000 UTC m=+24.936332665"
Jul 17 09:19:01.627767 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:01.627660    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T09:19:00.782435341Z","UUID":"8049709d-e2fe-447b-93fb-b93b8f2a7f81","Handler":null,"Name":"","Endpoint":""}
Jul 17 09:19:01.629337 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:01.629317    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 09:19:01.629497 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:01.629354    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 09:19:02.751364 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:02.751161    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:19:02.751835 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:02.751700    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:02.751835 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:02.751804    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:19:02.751941 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:02.751866    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:02.751941 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:02.751928    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:19:02.751941 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:02.751868    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:19:02.752087 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:02.752008    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:19:02.752087 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:02.752050    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:19:02.859574 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:02.859543    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz" event={"ID":"0f229870-274c-459a-8e97-c8328417069a","Type":"ContainerStarted","Data":"20e2d12051ffac41f514c8f184d260e9453986a8b587063a937f466d3c6e0bae"}
Jul 17 09:19:02.862281 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:02.862262    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:19:02.862656 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:02.862639    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq" event={"ID":"29ba670a-9f03-4942-be1e-79807bf25031","Type":"ContainerStarted","Data":"215b126dbe7c20bd67366de325e2f73c46291c1d5f6514becc1a55ca1aa0ead2"}
Jul 17 09:19:02.862717 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:02.862698    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:19:02.863184 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:02.863170    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hhg49"
Jul 17 09:19:02.909373 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:02.909338    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jxmmz" podStartSLOduration=3.458467325 podStartE2EDuration="24.909324528s" totalImagesPullingTime="21.450857203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:02.886393055 +0000 UTC m=+26.951213799" watchObservedRunningTime="2026-07-17 09:19:02.909324528 +0000 UTC m=+26.974145250"
Jul 17 09:19:04.752841 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:04.752661    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:04.753446 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:04.752702    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:04.753446 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:04.752927    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:19:04.753446 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:04.752702    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:19:04.753446 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:04.752992    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:19:04.753446 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:04.753102    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:19:04.871175 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:04.871145    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:19:04.871668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:04.871625    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq" event={"ID":"29ba670a-9f03-4942-be1e-79807bf25031","Type":"ContainerStarted","Data":"96a194fad43a3fe7953758ee1dc6d36b11d1da9ff93c2cb8e7e2bb5bef93e6a0"}
Jul 17 09:19:04.872072 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:04.872055    2571 scope.go:117] "RemoveContainer" containerID="87f89218da7744c17f5af1fe2c7bad7f9b0bba1b1487e085c4786d24a98f72e6"
Jul 17 09:19:06.470010 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.469980    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jlhnc"]
Jul 17 09:19:06.470399 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.470149    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:06.470399 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:06.470257    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:19:06.472915 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.472879    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5tsrt"]
Jul 17 09:19:06.473035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.473019    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:06.473121 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:06.473100    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:19:06.473512 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.473491    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sttnl"]
Jul 17 09:19:06.473618 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.473599    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:19:06.473725 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:06.473701    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:19:06.880995 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.880930    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:19:06.881347 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.881324    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq" event={"ID":"29ba670a-9f03-4942-be1e-79807bf25031","Type":"ContainerStarted","Data":"de3a0d95198c72843161630d767f7bcdea09c1d2c7997a7cae0f90f50c3bf5d4"}
Jul 17 09:19:06.881660 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.881640    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:19:06.881768 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.881716    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:19:06.881768 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.881767    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:19:06.883549 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.883521    2571 generic.go:356] "Generic (PLEG): container finished" podID="e7237e04-1a64-475d-bf54-112f171705fe" containerID="d981b8169d94845de1c9c5e41f1fbe9de7650fbbc7e3982b7fd749ed6acb88e2" exitCode=0
Jul 17 09:19:06.883757 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.883563    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x524x" event={"ID":"e7237e04-1a64-475d-bf54-112f171705fe","Type":"ContainerDied","Data":"d981b8169d94845de1c9c5e41f1fbe9de7650fbbc7e3982b7fd749ed6acb88e2"}
Jul 17 09:19:06.897512 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.897253    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:19:06.897707 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.897687    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:19:06.927020 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:06.926971    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq" podStartSLOduration=9.224288848 podStartE2EDuration="28.926958072s" totalImagesPullingTime="19.702669224s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:06.926515767 +0000 UTC m=+30.991336491" watchObservedRunningTime="2026-07-17 09:19:06.926958072 +0000 UTC m=+30.991778772"
Jul 17 09:19:07.752730 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:07.752694    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:19:07.753153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:07.752802    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:07.753153 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:07.752819    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:19:07.753153 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:07.752916    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:19:07.753153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:07.752950    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:07.753153 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:07.753024    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:19:08.890669 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:08.890636    2571 generic.go:356] "Generic (PLEG): container finished" podID="e7237e04-1a64-475d-bf54-112f171705fe" containerID="ec77c8171eeaa8585e14fa1b68346ee20866cdaa80440db625226941ed0a79c9" exitCode=0
Jul 17 09:19:08.891334 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:08.890725    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x524x" event={"ID":"e7237e04-1a64-475d-bf54-112f171705fe","Type":"ContainerDied","Data":"ec77c8171eeaa8585e14fa1b68346ee20866cdaa80440db625226941ed0a79c9"}
Jul 17 09:19:09.751910 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:09.751878    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:09.752082 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:09.751878    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:09.752082 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:09.752061    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:19:09.752082 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:09.751970    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:19:09.752241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:09.752129    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:19:09.752241 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:09.752217    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:19:09.895521 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:09.895426    2571 generic.go:356] "Generic (PLEG): container finished" podID="e7237e04-1a64-475d-bf54-112f171705fe" containerID="716a65cbf032d3c5532d0d633d8f51584930c96d65ace441f95ca64e0633d964" exitCode=0
Jul 17 09:19:09.895521 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:09.895493    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x524x" event={"ID":"e7237e04-1a64-475d-bf54-112f171705fe","Type":"ContainerDied","Data":"716a65cbf032d3c5532d0d633d8f51584930c96d65ace441f95ca64e0633d964"}
Jul 17 09:19:11.014731 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:11.014686    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:11.015202 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:11.014833    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:19:11.015202 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:11.014917    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret podName:0970881d-9524-4242-bc47-7f0813bf2df1 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:43.014896102 +0000 UTC m=+67.079716808 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret") pod "global-pull-secret-syncer-5tsrt" (UID: "0970881d-9524-4242-bc47-7f0813bf2df1") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:19:11.115760 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:11.115723    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:19:11.115760 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:11.115765    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzc4\" (UniqueName: \"kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4\") pod \"network-check-target-jlhnc\" (UID: \"0d2078c6-e7d7-495a-aa8f-45d4356cee4e\") " pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:11.115980 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:11.115893    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:19:11.115980 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:11.115964    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs podName:ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea nodeName:}" failed. No retries permitted until 2026-07-17 09:19:43.115948136 +0000 UTC m=+67.180768847 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs") pod "network-metrics-daemon-sttnl" (UID: "ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:19:11.116057 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:11.115901    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:19:11.116057 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:11.116005    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:19:11.116057 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:11.116018    2571 projected.go:196] Error preparing data for projected volume kube-api-access-qxzc4 for pod openshift-network-diagnostics/network-check-target-jlhnc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:19:11.116180 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:11.116059    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4 podName:0d2078c6-e7d7-495a-aa8f-45d4356cee4e nodeName:}" failed. No retries permitted until 2026-07-17 09:19:43.116048236 +0000 UTC m=+67.180868937 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxzc4" (UniqueName: "kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4") pod "network-check-target-jlhnc" (UID: "0d2078c6-e7d7-495a-aa8f-45d4356cee4e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:19:11.752598 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:11.752560    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:11.752598 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:11.752584    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:11.752798 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:11.752560    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:19:11.752798 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:11.752670    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jlhnc" podUID="0d2078c6-e7d7-495a-aa8f-45d4356cee4e"
Jul 17 09:19:11.752798 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:11.752754    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sttnl" podUID="ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea"
Jul 17 09:19:11.752888 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:11.752815    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5tsrt" podUID="0970881d-9524-4242-bc47-7f0813bf2df1"
Jul 17 09:19:12.285498 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.285411    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-208.ec2.internal" event="NodeReady"
Jul 17 09:19:12.285842 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.285583    2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 09:19:12.341580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.341528    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pvp8l"]
Jul 17 09:19:12.345017 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.344986    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l2h6h"]
Jul 17 09:19:12.345174 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.345115    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.347554 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.347528    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4mh5z\""
Jul 17 09:19:12.347554 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.347545    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 09:19:12.347753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.347659    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 09:19:12.348701 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.348675    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l2h6h"
Jul 17 09:19:12.351019 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.350999    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 09:19:12.351111 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.351082    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 09:19:12.351233 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.351194    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cdjx7\""
Jul 17 09:19:12.351233 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.351224    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 09:19:12.355082 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.355060    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l2h6h"]
Jul 17 09:19:12.355895 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.355874    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pvp8l"]
Jul 17 09:19:12.526216 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.526183    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-tmp-dir\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.526373 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.526241    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kg6r\" (UniqueName: \"kubernetes.io/projected/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-kube-api-access-5kg6r\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.526373 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.526298    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.526373 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.526341    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgppk\" (UniqueName: \"kubernetes.io/projected/b962f3cc-8103-449d-95af-efe0a5d84bbf-kube-api-access-sgppk\") pod \"ingress-canary-l2h6h\" (UID: \"b962f3cc-8103-449d-95af-efe0a5d84bbf\") " pod="openshift-ingress-canary/ingress-canary-l2h6h"
Jul 17 09:19:12.526482 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.526375    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b962f3cc-8103-449d-95af-efe0a5d84bbf-cert\") pod \"ingress-canary-l2h6h\" (UID: \"b962f3cc-8103-449d-95af-efe0a5d84bbf\") " pod="openshift-ingress-canary/ingress-canary-l2h6h"
Jul 17 09:19:12.526482 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.526399    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-config-volume\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.626716 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.626641    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgppk\" (UniqueName: \"kubernetes.io/projected/b962f3cc-8103-449d-95af-efe0a5d84bbf-kube-api-access-sgppk\") pod \"ingress-canary-l2h6h\" (UID: \"b962f3cc-8103-449d-95af-efe0a5d84bbf\") " pod="openshift-ingress-canary/ingress-canary-l2h6h"
Jul 17 09:19:12.626716 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.626689    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b962f3cc-8103-449d-95af-efe0a5d84bbf-cert\") pod \"ingress-canary-l2h6h\" (UID: \"b962f3cc-8103-449d-95af-efe0a5d84bbf\") " pod="openshift-ingress-canary/ingress-canary-l2h6h"
Jul 17 09:19:12.626716 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.626713    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-config-volume\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.626971 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.626740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-tmp-dir\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.626971 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.626795    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kg6r\" (UniqueName: \"kubernetes.io/projected/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-kube-api-access-5kg6r\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.626971 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.626837    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.626971 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:12.626940    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:19:12.627149 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:12.626994    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls podName:4db5b93e-8ebb-4d86-b45a-0bee6a6890f8 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:13.126976839 +0000 UTC m=+37.191797554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls") pod "dns-default-pvp8l" (UID: "4db5b93e-8ebb-4d86-b45a-0bee6a6890f8") : secret "dns-default-metrics-tls" not found
Jul 17 09:19:12.627297 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.627277    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-config-volume\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.627455 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.627422    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-tmp-dir\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.631657 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.631487    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b962f3cc-8103-449d-95af-efe0a5d84bbf-cert\") pod \"ingress-canary-l2h6h\" (UID: \"b962f3cc-8103-449d-95af-efe0a5d84bbf\") " pod="openshift-ingress-canary/ingress-canary-l2h6h"
Jul 17 09:19:12.638638 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.638614    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kg6r\" (UniqueName: \"kubernetes.io/projected/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-kube-api-access-5kg6r\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:12.639081 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.639057    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgppk\" (UniqueName: \"kubernetes.io/projected/b962f3cc-8103-449d-95af-efe0a5d84bbf-kube-api-access-sgppk\") pod \"ingress-canary-l2h6h\" (UID: \"b962f3cc-8103-449d-95af-efe0a5d84bbf\") " pod="openshift-ingress-canary/ingress-canary-l2h6h"
Jul 17 09:19:12.664837 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.664797    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l2h6h"
Jul 17 09:19:12.818513 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:12.818480    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l2h6h"]
Jul 17 09:19:13.131106 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:13.131070    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:13.131325 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:13.131258    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:19:13.131392 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:13.131351    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls podName:4db5b93e-8ebb-4d86-b45a-0bee6a6890f8 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:14.131325631 +0000 UTC m=+38.196146333 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls") pod "dns-default-pvp8l" (UID: "4db5b93e-8ebb-4d86-b45a-0bee6a6890f8") : secret "dns-default-metrics-tls" not found
Jul 17 09:19:13.751902 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:13.751865    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:19:13.752428 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:13.752033    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:13.752428 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:13.752037    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:13.756022 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:13.755992    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 09:19:13.756022 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:13.756005    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 09:19:13.756022 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:13.756015    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzpfx\""
Jul 17 09:19:13.756259 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:13.756040    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 09:19:13.756259 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:13.756062    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t2lf7\""
Jul 17 09:19:13.756259 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:13.755992    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 09:19:14.139283 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:14.139192    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:14.139435 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:14.139348    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:19:14.139513 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:14.139445    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls podName:4db5b93e-8ebb-4d86-b45a-0bee6a6890f8 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:16.139420518 +0000 UTC m=+40.204241223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls") pod "dns-default-pvp8l" (UID: "4db5b93e-8ebb-4d86-b45a-0bee6a6890f8") : secret "dns-default-metrics-tls" not found
Jul 17 09:19:14.352101 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:19:14.352074    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb962f3cc_8103_449d_95af_efe0a5d84bbf.slice/crio-a7417ec8c091f165e79bf3789ab0c7728619cba4f225adc3123360925d9751df WatchSource:0}: Error finding container a7417ec8c091f165e79bf3789ab0c7728619cba4f225adc3123360925d9751df: Status 404 returned error can't find the container with id a7417ec8c091f165e79bf3789ab0c7728619cba4f225adc3123360925d9751df
Jul 17 09:19:14.911738 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:14.911395    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x524x" event={"ID":"e7237e04-1a64-475d-bf54-112f171705fe","Type":"ContainerStarted","Data":"456cfc7bbfe40805a8df1929f8bf53bf6d98568b9302dc9de614738c3cbe1a4a"}
Jul 17 09:19:14.913089 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:14.913014    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l2h6h" event={"ID":"b962f3cc-8103-449d-95af-efe0a5d84bbf","Type":"ContainerStarted","Data":"a7417ec8c091f165e79bf3789ab0c7728619cba4f225adc3123360925d9751df"}
Jul 17 09:19:15.917843 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:15.917813    2571 generic.go:356] "Generic (PLEG): container finished" podID="e7237e04-1a64-475d-bf54-112f171705fe" containerID="456cfc7bbfe40805a8df1929f8bf53bf6d98568b9302dc9de614738c3cbe1a4a" exitCode=0
Jul 17 09:19:15.918196 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:15.917867    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x524x" event={"ID":"e7237e04-1a64-475d-bf54-112f171705fe","Type":"ContainerDied","Data":"456cfc7bbfe40805a8df1929f8bf53bf6d98568b9302dc9de614738c3cbe1a4a"}
Jul 17 09:19:16.156563 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:16.156521    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:16.156762 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:16.156668    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:19:16.156762 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:16.156730    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls podName:4db5b93e-8ebb-4d86-b45a-0bee6a6890f8 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:20.15671156 +0000 UTC m=+44.221532260 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls") pod "dns-default-pvp8l" (UID: "4db5b93e-8ebb-4d86-b45a-0bee6a6890f8") : secret "dns-default-metrics-tls" not found
Jul 17 09:19:16.922996 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:16.922967    2571 generic.go:356] "Generic (PLEG): container finished" podID="e7237e04-1a64-475d-bf54-112f171705fe" containerID="c681a6c8a7f72e69f8d944eef1b62b939544a3f6aba5b37a28975cdf77a201b2" exitCode=0
Jul 17 09:19:16.923416 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:16.923014    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x524x" event={"ID":"e7237e04-1a64-475d-bf54-112f171705fe","Type":"ContainerDied","Data":"c681a6c8a7f72e69f8d944eef1b62b939544a3f6aba5b37a28975cdf77a201b2"}
Jul 17 09:19:17.928289 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:17.928251    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-x524x" event={"ID":"e7237e04-1a64-475d-bf54-112f171705fe","Type":"ContainerStarted","Data":"1e78e055ce6d32231337f82ec48943aa834a3dcfb487dc7db3c70df6d08acf58"}
Jul 17 09:19:17.929719 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:17.929696    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l2h6h" event={"ID":"b962f3cc-8103-449d-95af-efe0a5d84bbf","Type":"ContainerStarted","Data":"8c17a1fd530fcbdcf62f4171004473a1de99bab03943eb28c154ca27ed8abea9"}
Jul 17 09:19:17.955738 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:17.955679    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-x524x" podStartSLOduration=7.241665111 podStartE2EDuration="39.955660252s" totalImagesPullingTime="32.713995141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:17.954056027 +0000 UTC m=+42.018876753" watchObservedRunningTime="2026-07-17 09:19:17.955660252 +0000 UTC m=+42.020480969"
Jul 17 09:19:17.970480 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:17.970419    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l2h6h" podStartSLOduration=3.097021369 podStartE2EDuration="5.970402312s" totalImagesPullingTime="2.873380943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:17.97012098 +0000 UTC m=+42.034941703" watchObservedRunningTime="2026-07-17 09:19:17.970402312 +0000 UTC m=+42.035223033"
Jul 17 09:19:20.185768 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:20.185733    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:20.186152 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:20.185855    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:19:20.186152 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:20.185903    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls podName:4db5b93e-8ebb-4d86-b45a-0bee6a6890f8 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:28.185889542 +0000 UTC m=+52.250710242 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls") pod "dns-default-pvp8l" (UID: "4db5b93e-8ebb-4d86-b45a-0bee6a6890f8") : secret "dns-default-metrics-tls" not found
Jul 17 09:19:28.235866 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:28.235823    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:28.236343 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:28.235956    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:19:28.236343 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:28.236017    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls podName:4db5b93e-8ebb-4d86-b45a-0bee6a6890f8 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:44.236002535 +0000 UTC m=+68.300823236 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls") pod "dns-default-pvp8l" (UID: "4db5b93e-8ebb-4d86-b45a-0bee6a6890f8") : secret "dns-default-metrics-tls" not found
Jul 17 09:19:30.766456 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.766418    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"]
Jul 17 09:19:30.790822 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.790796    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kssqm"]
Jul 17 09:19:30.790980 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.790914    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:30.795733 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.795710    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pmw5q\""
Jul 17 09:19:30.796754 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.796723    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 17 09:19:30.797900 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.797877    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 09:19:30.797999 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.797881    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 17 09:19:30.797999 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.797928    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 09:19:30.814828 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.814807    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kssqm"]
Jul 17 09:19:30.814947 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.814838    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"]
Jul 17 09:19:30.814947 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.814918    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kssqm"
Jul 17 09:19:30.817675 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.817657    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-hw6rx\""
Jul 17 09:19:30.853999 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.853965    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpvw8\" (UniqueName: \"kubernetes.io/projected/da993dbb-9c00-47e6-a410-8c8692e4644c-kube-api-access-zpvw8\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:30.854134 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.854019    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:30.854134 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.854071    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/da993dbb-9c00-47e6-a410-8c8692e4644c-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:30.854134 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.854102    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9b5m\" (UniqueName: \"kubernetes.io/projected/2f754948-6974-4233-b70c-a40088914fa0-kube-api-access-k9b5m\") pod \"network-check-source-7bc5b6bf85-kssqm\" (UID: \"2f754948-6974-4233-b70c-a40088914fa0\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kssqm"
Jul 17 09:19:30.913066 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.913033    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-nmj78"]
Jul 17 09:19:30.934088 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.934064    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-98q75"]
Jul 17 09:19:30.934230 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.934187    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-nmj78"
Jul 17 09:19:30.943621 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.943603    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 17 09:19:30.943729 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.943605    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:19:30.943729 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.943657    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bwkbk\""
Jul 17 09:19:30.955132 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.955108    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/da993dbb-9c00-47e6-a410-8c8692e4644c-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:30.955262 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.955159    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9b5m\" (UniqueName: \"kubernetes.io/projected/2f754948-6974-4233-b70c-a40088914fa0-kube-api-access-k9b5m\") pod \"network-check-source-7bc5b6bf85-kssqm\" (UID: \"2f754948-6974-4233-b70c-a40088914fa0\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kssqm"
Jul 17 09:19:30.955262 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.955204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpvw8\" (UniqueName: \"kubernetes.io/projected/da993dbb-9c00-47e6-a410-8c8692e4644c-kube-api-access-zpvw8\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:30.955262 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.955240    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:30.955423 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.955288    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq8bw\" (UniqueName: \"kubernetes.io/projected/bcfb2609-29dc-4ae7-ab3a-3a60d2334520-kube-api-access-hq8bw\") pod \"volume-data-source-validator-78cf4c6f6-nmj78\" (UID: \"bcfb2609-29dc-4ae7-ab3a-3a60d2334520\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-nmj78"
Jul 17 09:19:30.955423 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:30.955332    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:30.955423 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:30.955391    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls podName:da993dbb-9c00-47e6-a410-8c8692e4644c nodeName:}" failed. No retries permitted until 2026-07-17 09:19:31.455372994 +0000 UTC m=+55.520193694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jprcn" (UID: "da993dbb-9c00-47e6-a410-8c8692e4644c") : secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:30.955855 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.955837    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/da993dbb-9c00-47e6-a410-8c8692e4644c-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:30.956362 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.956346    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-nmj78"]
Jul 17 09:19:30.956480 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.956450    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:30.961539 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.961512    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 17 09:19:30.962020 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.961996    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-98q75"]
Jul 17 09:19:30.963418 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.963400    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 17 09:19:30.964344 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.964323    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 09:19:30.964593 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.964568    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 09:19:30.964696 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.964662    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f7qs5\""
Jul 17 09:19:30.967048 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.967031    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 17 09:19:30.983986 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.983961    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9b5m\" (UniqueName: \"kubernetes.io/projected/2f754948-6974-4233-b70c-a40088914fa0-kube-api-access-k9b5m\") pod \"network-check-source-7bc5b6bf85-kssqm\" (UID: \"2f754948-6974-4233-b70c-a40088914fa0\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kssqm"
Jul 17 09:19:30.987978 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:30.987953    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpvw8\" (UniqueName: \"kubernetes.io/projected/da993dbb-9c00-47e6-a410-8c8692e4644c-kube-api-access-zpvw8\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:31.003314 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.003288    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d596888c-k5kh6"]
Jul 17 09:19:31.018750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.018687    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.021499 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.021480    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 17 09:19:31.022613 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.022590    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7jzf9\""
Jul 17 09:19:31.022738 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.022594    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 17 09:19:31.022933 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.022919    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 17 09:19:31.026825 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.026809    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 17 09:19:31.029356 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.029334    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d596888c-k5kh6"]
Jul 17 09:19:31.056420 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056386    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96b39859-a448-4d17-aa25-35628ab46939-service-ca-bundle\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.056420 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056419    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrt4r\" (UniqueName: \"kubernetes.io/projected/96b39859-a448-4d17-aa25-35628ab46939-kube-api-access-qrt4r\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.056621 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056449    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b39859-a448-4d17-aa25-35628ab46939-tmp\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.056621 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056493    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-installation-pull-secrets\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.056621 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056588    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq8bw\" (UniqueName: \"kubernetes.io/projected/bcfb2609-29dc-4ae7-ab3a-3a60d2334520-kube-api-access-hq8bw\") pod \"volume-data-source-validator-78cf4c6f6-nmj78\" (UID: \"bcfb2609-29dc-4ae7-ab3a-3a60d2334520\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-nmj78"
Jul 17 09:19:31.056621 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056617    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-trusted-ca\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.056774 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056659    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/96b39859-a448-4d17-aa25-35628ab46939-serving-cert\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.056774 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056680    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/96b39859-a448-4d17-aa25-35628ab46939-snapshots\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.056774 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056758    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.056898 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056792    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-registry-certificates\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.056898 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056820    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-bound-sa-token\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.056898 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056864    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/394741d2-bf2f-4627-801d-8721cf6071c0-ca-trust-extracted\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.056985 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056915    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96b39859-a448-4d17-aa25-35628ab46939-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.056985 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056934    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-image-registry-private-configuration\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.056985 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.056972    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq749\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-kube-api-access-cq749\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.069376 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.069353    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq8bw\" (UniqueName: \"kubernetes.io/projected/bcfb2609-29dc-4ae7-ab3a-3a60d2334520-kube-api-access-hq8bw\") pod \"volume-data-source-validator-78cf4c6f6-nmj78\" (UID: \"bcfb2609-29dc-4ae7-ab3a-3a60d2334520\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-nmj78"
Jul 17 09:19:31.123328 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.123288    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kssqm"
Jul 17 09:19:31.158067 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158040    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-trusted-ca\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.158170 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158092    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/96b39859-a448-4d17-aa25-35628ab46939-serving-cert\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.158241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158219    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/96b39859-a448-4d17-aa25-35628ab46939-snapshots\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.158301 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158268    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.158353 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158298    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-registry-certificates\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.158353 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158326    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-bound-sa-token\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.158449 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158357    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/394741d2-bf2f-4627-801d-8721cf6071c0-ca-trust-extracted\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.158449 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:31.158424    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:19:31.158449 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:31.158443    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d596888c-k5kh6: secret "image-registry-tls" not found
Jul 17 09:19:31.158619 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:31.158541    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls podName:394741d2-bf2f-4627-801d-8721cf6071c0 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:31.658520888 +0000 UTC m=+55.723341600 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls") pod "image-registry-6d596888c-k5kh6" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0") : secret "image-registry-tls" not found
Jul 17 09:19:31.158619 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158589    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96b39859-a448-4d17-aa25-35628ab46939-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.158730 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158616    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-image-registry-private-configuration\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.158730 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158663    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq749\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-kube-api-access-cq749\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.158730 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158702    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96b39859-a448-4d17-aa25-35628ab46939-service-ca-bundle\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.158911 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158727    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrt4r\" (UniqueName: \"kubernetes.io/projected/96b39859-a448-4d17-aa25-35628ab46939-kube-api-access-qrt4r\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.158911 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158730    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/394741d2-bf2f-4627-801d-8721cf6071c0-ca-trust-extracted\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.158911 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158762    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b39859-a448-4d17-aa25-35628ab46939-tmp\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.158911 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158791    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-installation-pull-secrets\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.159095 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.158932    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/96b39859-a448-4d17-aa25-35628ab46939-snapshots\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.159928 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.159335    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-registry-certificates\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.159928 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.159605    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-trusted-ca\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.159928 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.159869    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b39859-a448-4d17-aa25-35628ab46939-tmp\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.160117 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.159969    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96b39859-a448-4d17-aa25-35628ab46939-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.160164 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.160143    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96b39859-a448-4d17-aa25-35628ab46939-service-ca-bundle\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.160868 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.160839    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/96b39859-a448-4d17-aa25-35628ab46939-serving-cert\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.161352 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.161331    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-installation-pull-secrets\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.161430 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.161385    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-image-registry-private-configuration\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.174101 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.174082    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-bound-sa-token\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.174247 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.174226    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq749\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-kube-api-access-cq749\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.174360 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.174344    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrt4r\" (UniqueName: \"kubernetes.io/projected/96b39859-a448-4d17-aa25-35628ab46939-kube-api-access-qrt4r\") pod \"insights-operator-555d9d7885-98q75\" (UID: \"96b39859-a448-4d17-aa25-35628ab46939\") " pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.242936 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.242906    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-nmj78"
Jul 17 09:19:31.266927 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.266901    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kssqm"]
Jul 17 09:19:31.267937 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.267918    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-98q75"
Jul 17 09:19:31.270092 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:19:31.270064    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f754948_6974_4233_b70c_a40088914fa0.slice/crio-03ca9e8eb5dd74fe37ded94681448e9377e015257e3d93e3b555aa0388f0ae07 WatchSource:0}: Error finding container 03ca9e8eb5dd74fe37ded94681448e9377e015257e3d93e3b555aa0388f0ae07: Status 404 returned error can't find the container with id 03ca9e8eb5dd74fe37ded94681448e9377e015257e3d93e3b555aa0388f0ae07
Jul 17 09:19:31.405763 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.405730    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-nmj78"]
Jul 17 09:19:31.408967 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:19:31.408936    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcfb2609_29dc_4ae7_ab3a_3a60d2334520.slice/crio-9d263a10b999ea51364a5e2b16a87b351fcb7bef11a79abe60c8ab4f7119ee8e WatchSource:0}: Error finding container 9d263a10b999ea51364a5e2b16a87b351fcb7bef11a79abe60c8ab4f7119ee8e: Status 404 returned error can't find the container with id 9d263a10b999ea51364a5e2b16a87b351fcb7bef11a79abe60c8ab4f7119ee8e
Jul 17 09:19:31.425864 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.425838    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-98q75"]
Jul 17 09:19:31.429792 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:19:31.429762    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b39859_a448_4d17_aa25_35628ab46939.slice/crio-9d32bb70cb00f4e5cabe2a44e0933cb7799fc460562de90ec6a35b8128a605cd WatchSource:0}: Error finding container 9d32bb70cb00f4e5cabe2a44e0933cb7799fc460562de90ec6a35b8128a605cd: Status 404 returned error can't find the container with id 9d32bb70cb00f4e5cabe2a44e0933cb7799fc460562de90ec6a35b8128a605cd
Jul 17 09:19:31.462019 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.461979    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:31.462189 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:31.462125    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:31.462237 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:31.462195    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls podName:da993dbb-9c00-47e6-a410-8c8692e4644c nodeName:}" failed. No retries permitted until 2026-07-17 09:19:32.462177231 +0000 UTC m=+56.526997932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jprcn" (UID: "da993dbb-9c00-47e6-a410-8c8692e4644c") : secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:31.663494 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.663402    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:31.663653 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:31.663563    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:19:31.663653 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:31.663584    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d596888c-k5kh6: secret "image-registry-tls" not found
Jul 17 09:19:31.663653 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:31.663642    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls podName:394741d2-bf2f-4627-801d-8721cf6071c0 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:32.663623737 +0000 UTC m=+56.728444437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls") pod "image-registry-6d596888c-k5kh6" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0") : secret "image-registry-tls" not found
Jul 17 09:19:31.962655 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.962552    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-nmj78" event={"ID":"bcfb2609-29dc-4ae7-ab3a-3a60d2334520","Type":"ContainerStarted","Data":"9d263a10b999ea51364a5e2b16a87b351fcb7bef11a79abe60c8ab4f7119ee8e"}
Jul 17 09:19:31.964863 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.964783    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kssqm" event={"ID":"2f754948-6974-4233-b70c-a40088914fa0","Type":"ContainerStarted","Data":"03ca9e8eb5dd74fe37ded94681448e9377e015257e3d93e3b555aa0388f0ae07"}
Jul 17 09:19:31.967345 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:31.967291    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-98q75" event={"ID":"96b39859-a448-4d17-aa25-35628ab46939","Type":"ContainerStarted","Data":"9d32bb70cb00f4e5cabe2a44e0933cb7799fc460562de90ec6a35b8128a605cd"}
Jul 17 09:19:32.470423 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:32.470379    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:32.470589 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:32.470553    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:32.470630 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:32.470623    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls podName:da993dbb-9c00-47e6-a410-8c8692e4644c nodeName:}" failed. No retries permitted until 2026-07-17 09:19:34.470606791 +0000 UTC m=+58.535427491 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jprcn" (UID: "da993dbb-9c00-47e6-a410-8c8692e4644c") : secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:32.672238 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:32.672194    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:32.672430 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:32.672381    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:19:32.672430 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:32.672403    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d596888c-k5kh6: secret "image-registry-tls" not found
Jul 17 09:19:32.672589 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:32.672491    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls podName:394741d2-bf2f-4627-801d-8721cf6071c0 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:34.672453689 +0000 UTC m=+58.737274390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls") pod "image-registry-6d596888c-k5kh6" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0") : secret "image-registry-tls" not found
Jul 17 09:19:32.972160 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:32.972107    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-nmj78" event={"ID":"bcfb2609-29dc-4ae7-ab3a-3a60d2334520","Type":"ContainerStarted","Data":"232ab5dadd36790b2ccfcb5f57b9bd8b6bbe38545e4934708c8b71788639a2d6"}
Jul 17 09:19:32.996487 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:32.996356    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-nmj78" podStartSLOduration=1.664191658 podStartE2EDuration="2.996338225s" totalImagesPullingTime="1.332146567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:32.995895011 +0000 UTC m=+57.060715735" watchObservedRunningTime="2026-07-17 09:19:32.996338225 +0000 UTC m=+57.061158944"
Jul 17 09:19:33.976847 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:33.976803    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-98q75" event={"ID":"96b39859-a448-4d17-aa25-35628ab46939","Type":"ContainerStarted","Data":"7d6c0f9aafba63768c9df528a2a7f14d0c53c31b90cdc007f2de19f08b2eaa65"}
Jul 17 09:19:34.002252 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.002170    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-98q75" podStartSLOduration=2.020462099 podStartE2EDuration="4.002152843s" totalImagesPullingTime="1.981690744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:34.001695089 +0000 UTC m=+58.066515810" watchObservedRunningTime="2026-07-17 09:19:34.002152843 +0000 UTC m=+58.066973565"
Jul 17 09:19:34.297899 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.297819    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-48h52"]
Jul 17 09:19:34.301438 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.301409    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.305602 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.305573    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 09:19:34.305735 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.305601    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cb5vz\""
Jul 17 09:19:34.305735 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.305583    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 09:19:34.348172 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.348077    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-48h52"]
Jul 17 09:19:34.387771 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.387737    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.387939 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.387804    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.387939 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.387925    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d8v2\" (UniqueName: \"kubernetes.io/projected/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-api-access-6d8v2\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.388074 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.387975    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-data-volume\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.388074 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.388038    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-crio-socket\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.489252 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.489217    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.489425 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.489279    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.489425 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.489312    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:34.489425 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.489337    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d8v2\" (UniqueName: \"kubernetes.io/projected/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-api-access-6d8v2\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.489425 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.489380    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-data-volume\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.489425 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.489398    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:34.489425 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.489424    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-crio-socket\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.489839 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.489437    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:19:34.489839 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.489488    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls podName:bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb nodeName:}" failed. No retries permitted until 2026-07-17 09:19:34.989452994 +0000 UTC m=+59.054273719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-48h52" (UID: "bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:34.489839 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.489498    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:34.489839 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.489526    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-crio-socket\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.489839 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.489537    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm podName:bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb nodeName:}" failed. No retries permitted until 2026-07-17 09:19:34.98951673 +0000 UTC m=+59.054337430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-48h52" (UID: "bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb") : configmap "kube-rbac-proxy" not found
Jul 17 09:19:34.489839 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.489559    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls podName:da993dbb-9c00-47e6-a410-8c8692e4644c nodeName:}" failed. No retries permitted until 2026-07-17 09:19:38.489549145 +0000 UTC m=+62.554369846 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jprcn" (UID: "da993dbb-9c00-47e6-a410-8c8692e4644c") : secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:34.489839 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.489809    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-data-volume\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.503791 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.503764    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d8v2\" (UniqueName: \"kubernetes.io/projected/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-api-access-6d8v2\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.692119 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.691573    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:34.692119 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.691869    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:19:34.692119 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.691887    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d596888c-k5kh6: secret "image-registry-tls" not found
Jul 17 09:19:34.692119 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.691943    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls podName:394741d2-bf2f-4627-801d-8721cf6071c0 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:38.691925029 +0000 UTC m=+62.756745732 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls") pod "image-registry-6d596888c-k5kh6" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0") : secret "image-registry-tls" not found
Jul 17 09:19:34.980304 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.980266    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kssqm" event={"ID":"2f754948-6974-4233-b70c-a40088914fa0","Type":"ContainerStarted","Data":"32138b422e1f68a0edb6eee270962751dbc7ef7dda8f17e3f72a772a23e7e0f7"}
Jul 17 09:19:34.993748 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.993721    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.993855 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:34.993825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:34.993891 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.993856    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:19:34.993936 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.993925    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm podName:bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb nodeName:}" failed. No retries permitted until 2026-07-17 09:19:35.993909274 +0000 UTC m=+60.058729978 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-48h52" (UID: "bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb") : configmap "kube-rbac-proxy" not found
Jul 17 09:19:34.993936 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.993932    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:34.994004 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:34.993969    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls podName:bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb nodeName:}" failed. No retries permitted until 2026-07-17 09:19:35.993958385 +0000 UTC m=+60.058779085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-48h52" (UID: "bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:35.004252 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:35.004195    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kssqm" podStartSLOduration=1.758623034 podStartE2EDuration="5.004174056s" totalImagesPullingTime="3.245551022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:35.00326688 +0000 UTC m=+59.068087601" watchObservedRunningTime="2026-07-17 09:19:35.004174056 +0000 UTC m=+59.068994779"
Jul 17 09:19:36.003255 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.003192    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:36.003255 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.003264    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:36.003769 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:36.003354    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:19:36.003769 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:36.003372    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:36.003769 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:36.003409    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm podName:bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb nodeName:}" failed. No retries permitted until 2026-07-17 09:19:38.003395933 +0000 UTC m=+62.068216633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-48h52" (UID: "bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb") : configmap "kube-rbac-proxy" not found
Jul 17 09:19:36.003769 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:36.003444    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls podName:bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb nodeName:}" failed. No retries permitted until 2026-07-17 09:19:38.00343032 +0000 UTC m=+62.068251020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-48h52" (UID: "bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:36.210754 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.210717    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj"]
Jul 17 09:19:36.212659 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.212643    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj"
Jul 17 09:19:36.216362 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.216339    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-pb4pj\""
Jul 17 09:19:36.216504 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.216339    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 17 09:19:36.217563 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.217543    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 17 09:19:36.239661 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.239626    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj"]
Jul 17 09:19:36.281367 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.281296    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n5hmq_199a5035-60c1-4b0d-941e-4a1a20291037/dns-node-resolver/0.log"
Jul 17 09:19:36.306288 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.306252    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgr5c\" (UniqueName: \"kubernetes.io/projected/debff7eb-0899-41e0-9202-d1dfd5d3a375-kube-api-access-jgr5c\") pod \"migrator-7cc9c7ffd7-hkvkj\" (UID: \"debff7eb-0899-41e0-9202-d1dfd5d3a375\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj"
Jul 17 09:19:36.406652 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.406610    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgr5c\" (UniqueName: \"kubernetes.io/projected/debff7eb-0899-41e0-9202-d1dfd5d3a375-kube-api-access-jgr5c\") pod \"migrator-7cc9c7ffd7-hkvkj\" (UID: \"debff7eb-0899-41e0-9202-d1dfd5d3a375\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj"
Jul 17 09:19:36.416980 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.416952    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgr5c\" (UniqueName: \"kubernetes.io/projected/debff7eb-0899-41e0-9202-d1dfd5d3a375-kube-api-access-jgr5c\") pod \"migrator-7cc9c7ffd7-hkvkj\" (UID: \"debff7eb-0899-41e0-9202-d1dfd5d3a375\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj"
Jul 17 09:19:36.521282 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.521233    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj"
Jul 17 09:19:36.654457 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.654424    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj"]
Jul 17 09:19:36.659330 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:19:36.659291    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddebff7eb_0899_41e0_9202_d1dfd5d3a375.slice/crio-8f58bf5caff26a64342aed6c13347df6e47b1c95da0ea48f9614c67bf1a3c1bf WatchSource:0}: Error finding container 8f58bf5caff26a64342aed6c13347df6e47b1c95da0ea48f9614c67bf1a3c1bf: Status 404 returned error can't find the container with id 8f58bf5caff26a64342aed6c13347df6e47b1c95da0ea48f9614c67bf1a3c1bf
Jul 17 09:19:36.986571 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:36.986537    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj" event={"ID":"debff7eb-0899-41e0-9202-d1dfd5d3a375","Type":"ContainerStarted","Data":"8f58bf5caff26a64342aed6c13347df6e47b1c95da0ea48f9614c67bf1a3c1bf"}
Jul 17 09:19:37.259310 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.259234    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ksfkn_53816944-d4bc-4e11-8c58-b9427233583a/node-ca/0.log"
Jul 17 09:19:37.390651 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.390616    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-ddbwh"]
Jul 17 09:19:37.395480 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.395449    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-ddbwh"
Jul 17 09:19:37.398441 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.398410    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 17 09:19:37.398593 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.398574    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 17 09:19:37.399617 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.399593    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 17 09:19:37.399728 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.399595    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 17 09:19:37.399854 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.399840    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qcbpb\""
Jul 17 09:19:37.408119 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.408093    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-ddbwh"]
Jul 17 09:19:37.516624 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.516549    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6abd44b5-688c-45dd-a0cf-8169438c88f9-signing-key\") pod \"service-ca-58598695f7-ddbwh\" (UID: \"6abd44b5-688c-45dd-a0cf-8169438c88f9\") " pod="openshift-service-ca/service-ca-58598695f7-ddbwh"
Jul 17 09:19:37.516624 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.516586    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4fjw\" (UniqueName: \"kubernetes.io/projected/6abd44b5-688c-45dd-a0cf-8169438c88f9-kube-api-access-z4fjw\") pod \"service-ca-58598695f7-ddbwh\" (UID: \"6abd44b5-688c-45dd-a0cf-8169438c88f9\") " pod="openshift-service-ca/service-ca-58598695f7-ddbwh"
Jul 17 09:19:37.516835 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.516688    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6abd44b5-688c-45dd-a0cf-8169438c88f9-signing-cabundle\") pod \"service-ca-58598695f7-ddbwh\" (UID: \"6abd44b5-688c-45dd-a0cf-8169438c88f9\") " pod="openshift-service-ca/service-ca-58598695f7-ddbwh"
Jul 17 09:19:37.617269 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.617225    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6abd44b5-688c-45dd-a0cf-8169438c88f9-signing-key\") pod \"service-ca-58598695f7-ddbwh\" (UID: \"6abd44b5-688c-45dd-a0cf-8169438c88f9\") " pod="openshift-service-ca/service-ca-58598695f7-ddbwh"
Jul 17 09:19:37.617269 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.617268    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4fjw\" (UniqueName: \"kubernetes.io/projected/6abd44b5-688c-45dd-a0cf-8169438c88f9-kube-api-access-z4fjw\") pod \"service-ca-58598695f7-ddbwh\" (UID: \"6abd44b5-688c-45dd-a0cf-8169438c88f9\") " pod="openshift-service-ca/service-ca-58598695f7-ddbwh"
Jul 17 09:19:37.617495 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.617391    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6abd44b5-688c-45dd-a0cf-8169438c88f9-signing-cabundle\") pod \"service-ca-58598695f7-ddbwh\" (UID: \"6abd44b5-688c-45dd-a0cf-8169438c88f9\") " pod="openshift-service-ca/service-ca-58598695f7-ddbwh"
Jul 17 09:19:37.618145 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.618113    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6abd44b5-688c-45dd-a0cf-8169438c88f9-signing-cabundle\") pod \"service-ca-58598695f7-ddbwh\" (UID: \"6abd44b5-688c-45dd-a0cf-8169438c88f9\") " pod="openshift-service-ca/service-ca-58598695f7-ddbwh"
Jul 17 09:19:37.619963 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.619938    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6abd44b5-688c-45dd-a0cf-8169438c88f9-signing-key\") pod \"service-ca-58598695f7-ddbwh\" (UID: \"6abd44b5-688c-45dd-a0cf-8169438c88f9\") " pod="openshift-service-ca/service-ca-58598695f7-ddbwh"
Jul 17 09:19:37.628715 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.628688    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4fjw\" (UniqueName: \"kubernetes.io/projected/6abd44b5-688c-45dd-a0cf-8169438c88f9-kube-api-access-z4fjw\") pod \"service-ca-58598695f7-ddbwh\" (UID: \"6abd44b5-688c-45dd-a0cf-8169438c88f9\") " pod="openshift-service-ca/service-ca-58598695f7-ddbwh"
Jul 17 09:19:37.707482 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.707438    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-ddbwh"
Jul 17 09:19:37.928229 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:37.928197    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-ddbwh"]
Jul 17 09:19:38.019569 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:38.019531    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:38.019769 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:38.019654    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:38.019769 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:38.019677    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:19:38.019769 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:38.019767    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm podName:bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb nodeName:}" failed. No retries permitted until 2026-07-17 09:19:42.019744126 +0000 UTC m=+66.084564827 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-48h52" (UID: "bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb") : configmap "kube-rbac-proxy" not found
Jul 17 09:19:38.019940 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:38.019778    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:38.019940 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:38.019830    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls podName:bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb nodeName:}" failed. No retries permitted until 2026-07-17 09:19:42.019816523 +0000 UTC m=+66.084637223 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-48h52" (UID: "bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:38.057962 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:38.057890    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l2h6h_b962f3cc-8103-449d-95af-efe0a5d84bbf/serve-healthcheck-canary/0.log"
Jul 17 09:19:38.098730 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:19:38.098693    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6abd44b5_688c_45dd_a0cf_8169438c88f9.slice/crio-353669b736966503bc1dd388c84468bb5686357088e58c807d5ec0bcadb8ebb5 WatchSource:0}: Error finding container 353669b736966503bc1dd388c84468bb5686357088e58c807d5ec0bcadb8ebb5: Status 404 returned error can't find the container with id 353669b736966503bc1dd388c84468bb5686357088e58c807d5ec0bcadb8ebb5
Jul 17 09:19:38.524794 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:38.524750    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:38.525226 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:38.524904    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:38.525226 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:38.524970    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls podName:da993dbb-9c00-47e6-a410-8c8692e4644c nodeName:}" failed. No retries permitted until 2026-07-17 09:19:46.524955162 +0000 UTC m=+70.589775861 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jprcn" (UID: "da993dbb-9c00-47e6-a410-8c8692e4644c") : secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:38.726144 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:38.726107    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:38.726309 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:38.726287    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:19:38.726309 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:38.726305    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d596888c-k5kh6: secret "image-registry-tls" not found
Jul 17 09:19:38.726387 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:38.726368    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls podName:394741d2-bf2f-4627-801d-8721cf6071c0 nodeName:}" failed. No retries permitted until 2026-07-17 09:19:46.726347222 +0000 UTC m=+70.791167922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls") pod "image-registry-6d596888c-k5kh6" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0") : secret "image-registry-tls" not found
Jul 17 09:19:38.908143 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:38.908109    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jhvwq"
Jul 17 09:19:38.996268 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:38.996186    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj" event={"ID":"debff7eb-0899-41e0-9202-d1dfd5d3a375","Type":"ContainerStarted","Data":"edd1d98f5b522d8bfb4da7df1a00f4b8fa1b181cbfe69d4c0efd6118f1b6ab0d"}
Jul 17 09:19:38.996268 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:38.996248    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj" event={"ID":"debff7eb-0899-41e0-9202-d1dfd5d3a375","Type":"ContainerStarted","Data":"f9a026be0ad68bb222b3ae1e4acdaf3f347c97abc7ed0517fbb1d8763305c544"}
Jul 17 09:19:38.997371 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:38.997346    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-ddbwh" event={"ID":"6abd44b5-688c-45dd-a0cf-8169438c88f9","Type":"ContainerStarted","Data":"353669b736966503bc1dd388c84468bb5686357088e58c807d5ec0bcadb8ebb5"}
Jul 17 09:19:39.020769 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:39.020706    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-hkvkj" podStartSLOduration=1.552154014 podStartE2EDuration="3.020685431s" totalImagesPullingTime="1.468531417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:39.018947513 +0000 UTC m=+63.083768237" watchObservedRunningTime="2026-07-17 09:19:39.020685431 +0000 UTC m=+63.085506151"
Jul 17 09:19:41.007079 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:41.007045    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-ddbwh" event={"ID":"6abd44b5-688c-45dd-a0cf-8169438c88f9","Type":"ContainerStarted","Data":"7db654669352e0218fc5a0dbb52d44a70e43c21c70d8c02f87e8cda0b391bfd9"}
Jul 17 09:19:41.040237 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:41.040182    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-ddbwh" podStartSLOduration=2.122816901 podStartE2EDuration="4.040167424s" totalImagesPullingTime="1.917350523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:41.039168704 +0000 UTC m=+65.103989439" watchObservedRunningTime="2026-07-17 09:19:41.040167424 +0000 UTC m=+65.104988146"
Jul 17 09:19:42.055335 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:42.055296    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:42.055863 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:42.055432    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:42.055863 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:42.055432    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:19:42.055863 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:42.055541    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:42.055863 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:42.055558    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm podName:bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb nodeName:}" failed. No retries permitted until 2026-07-17 09:19:50.055534071 +0000 UTC m=+74.120354775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-48h52" (UID: "bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb") : configmap "kube-rbac-proxy" not found
Jul 17 09:19:42.055863 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:42.055596    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls podName:bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb nodeName:}" failed. No retries permitted until 2026-07-17 09:19:50.055580556 +0000 UTC m=+74.120401256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-48h52" (UID: "bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:43.065261 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.065204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:43.068386 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.068366    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 09:19:43.078120 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.078086    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0970881d-9524-4242-bc47-7f0813bf2df1-original-pull-secret\") pod \"global-pull-secret-syncer-5tsrt\" (UID: \"0970881d-9524-4242-bc47-7f0813bf2df1\") " pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:43.165634 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.165597    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxzc4\" (UniqueName: \"kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4\") pod \"network-check-target-jlhnc\" (UID: \"0d2078c6-e7d7-495a-aa8f-45d4356cee4e\") " pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:43.165803 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.165662    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:19:43.168277 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.168249    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxzc4\" (UniqueName: \"kubernetes.io/projected/0d2078c6-e7d7-495a-aa8f-45d4356cee4e-kube-api-access-qxzc4\") pod \"network-check-target-jlhnc\" (UID: \"0d2078c6-e7d7-495a-aa8f-45d4356cee4e\") " pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:43.168657 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.168640    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 09:19:43.176224 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:43.176200    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 17 09:19:43.176342 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:43.176263    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs podName:ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea nodeName:}" failed. No retries permitted until 2026-07-17 09:20:47.176247706 +0000 UTC m=+131.241068406 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs") pod "network-metrics-daemon-sttnl" (UID: "ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea") : secret "metrics-daemon-secret" not found
Jul 17 09:19:43.176452 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.176434    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzpfx\""
Jul 17 09:19:43.179746 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.179732    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5tsrt"
Jul 17 09:19:43.184158 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.184141    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:43.319976 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.319870    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jlhnc"]
Jul 17 09:19:43.322759 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:19:43.322734    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d2078c6_e7d7_495a_aa8f_45d4356cee4e.slice/crio-975fa852df034d8594103962aa705d818326e3ff1126465cfda8c0eca0aee62a WatchSource:0}: Error finding container 975fa852df034d8594103962aa705d818326e3ff1126465cfda8c0eca0aee62a: Status 404 returned error can't find the container with id 975fa852df034d8594103962aa705d818326e3ff1126465cfda8c0eca0aee62a
Jul 17 09:19:43.334423 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:43.334398    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5tsrt"]
Jul 17 09:19:43.337099 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:19:43.337071    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0970881d_9524_4242_bc47_7f0813bf2df1.slice/crio-78d51b46869e23ef08dd144ca1a4749e13b4e48b786aad0b4531634b55ef67d7 WatchSource:0}: Error finding container 78d51b46869e23ef08dd144ca1a4749e13b4e48b786aad0b4531634b55ef67d7: Status 404 returned error can't find the container with id 78d51b46869e23ef08dd144ca1a4749e13b4e48b786aad0b4531634b55ef67d7
Jul 17 09:19:44.022577 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:44.022057    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jlhnc" event={"ID":"0d2078c6-e7d7-495a-aa8f-45d4356cee4e","Type":"ContainerStarted","Data":"8711905bb9379c70d3ec664315a37b6313a86f1e7d1f9db8102aaa0b36d9e8a4"}
Jul 17 09:19:44.022577 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:44.022119    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jlhnc" event={"ID":"0d2078c6-e7d7-495a-aa8f-45d4356cee4e","Type":"ContainerStarted","Data":"975fa852df034d8594103962aa705d818326e3ff1126465cfda8c0eca0aee62a"}
Jul 17 09:19:44.022577 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:44.022178    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:19:44.023725 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:44.023665    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5tsrt" event={"ID":"0970881d-9524-4242-bc47-7f0813bf2df1","Type":"ContainerStarted","Data":"78d51b46869e23ef08dd144ca1a4749e13b4e48b786aad0b4531634b55ef67d7"}
Jul 17 09:19:44.041725 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:44.041666    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jlhnc" podStartSLOduration=66.041645233 podStartE2EDuration="1m6.041645233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:44.040985628 +0000 UTC m=+68.105806353" watchObservedRunningTime="2026-07-17 09:19:44.041645233 +0000 UTC m=+68.106465959"
Jul 17 09:19:44.274792 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:44.274701    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:19:44.275196 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:44.274857    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:19:44.275196 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:44.274916    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls podName:4db5b93e-8ebb-4d86-b45a-0bee6a6890f8 nodeName:}" failed. No retries permitted until 2026-07-17 09:20:16.274899021 +0000 UTC m=+100.339719745 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls") pod "dns-default-pvp8l" (UID: "4db5b93e-8ebb-4d86-b45a-0bee6a6890f8") : secret "dns-default-metrics-tls" not found
Jul 17 09:19:46.594032 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:46.593987    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:19:46.594454 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:46.594139    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:46.594454 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:46.594219    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls podName:da993dbb-9c00-47e6-a410-8c8692e4644c nodeName:}" failed. No retries permitted until 2026-07-17 09:20:02.594198558 +0000 UTC m=+86.659019289 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jprcn" (UID: "da993dbb-9c00-47e6-a410-8c8692e4644c") : secret "cluster-monitoring-operator-tls" not found
Jul 17 09:19:46.795928 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:46.795864    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:46.798563 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:46.798535    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls\") pod \"image-registry-6d596888c-k5kh6\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") " pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:46.931815 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:46.931703    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7jzf9\""
Jul 17 09:19:46.939089 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:46.939056    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:48.079761 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:48.079734    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d596888c-k5kh6"]
Jul 17 09:19:48.084621 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:19:48.084595    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod394741d2_bf2f_4627_801d_8721cf6071c0.slice/crio-715a562ef6a8c5341eea7195e72f31488cd0d25371cbd38555dcc7c1006c3a9a WatchSource:0}: Error finding container 715a562ef6a8c5341eea7195e72f31488cd0d25371cbd38555dcc7c1006c3a9a: Status 404 returned error can't find the container with id 715a562ef6a8c5341eea7195e72f31488cd0d25371cbd38555dcc7c1006c3a9a
Jul 17 09:19:49.042852 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:49.042818    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d596888c-k5kh6" event={"ID":"394741d2-bf2f-4627-801d-8721cf6071c0","Type":"ContainerStarted","Data":"088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1"}
Jul 17 09:19:49.043037 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:49.042868    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d596888c-k5kh6" event={"ID":"394741d2-bf2f-4627-801d-8721cf6071c0","Type":"ContainerStarted","Data":"715a562ef6a8c5341eea7195e72f31488cd0d25371cbd38555dcc7c1006c3a9a"}
Jul 17 09:19:49.043037 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:49.042929    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:19:49.044271 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:49.044248    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5tsrt" event={"ID":"0970881d-9524-4242-bc47-7f0813bf2df1","Type":"ContainerStarted","Data":"47d64bb40e8a01ce899c74ddc5c856a993ce6ecb7ca06064e5ab952461906255"}
Jul 17 09:19:49.073689 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:49.073643    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d596888c-k5kh6" podStartSLOduration=19.073627453 podStartE2EDuration="19.073627453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:49.07312219 +0000 UTC m=+73.137942912" watchObservedRunningTime="2026-07-17 09:19:49.073627453 +0000 UTC m=+73.138448174"
Jul 17 09:19:50.124957 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:50.124920    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:50.125435 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:50.125030    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:19:50.125435 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:50.125072    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:50.125435 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:19:50.125162    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls podName:bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb nodeName:}" failed. No retries permitted until 2026-07-17 09:20:06.125140576 +0000 UTC m=+90.189961286 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-48h52" (UID: "bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:19:50.125675 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:19:50.125652    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:20:02.147597 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.147452    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5tsrt" podStartSLOduration=79.519442852 podStartE2EDuration="1m24.147437435s" totalImagesPullingTime="4.627994583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:19:49.09264115 +0000 UTC m=+73.157461872" watchObservedRunningTime="2026-07-17 09:20:02.147437435 +0000 UTC m=+86.212258165"
Jul 17 09:20:02.148085 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.148050    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d596888c-k5kh6"]
Jul 17 09:20:02.161682 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.161659    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5"]
Jul 17 09:20:02.165201 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.165185    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5"
Jul 17 09:20:02.170594 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.170568    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 17 09:20:02.170705 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.170627    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hxg5b\""
Jul 17 09:20:02.170840 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.170825    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 17 09:20:02.185289 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.185268    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5"]
Jul 17 09:20:02.272312 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.272282    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-5fnqc"]
Jul 17 09:20:02.274124 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.274108    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-5fnqc"
Jul 17 09:20:02.279847 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.279812    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 09:20:02.279847 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.279845    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 09:20:02.280048 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.279875    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ljr8v\""
Jul 17 09:20:02.281916 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.281896    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79b9779f5d-fzv2c"]
Jul 17 09:20:02.283647 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.283631    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.300081 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.300061    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-5fnqc"]
Jul 17 09:20:02.320630 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.320600    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79b9779f5d-fzv2c"]
Jul 17 09:20:02.323472 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.323436    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7452d789-b8dd-4ccc-b528-84ece1143cf9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9pmr5\" (UID: \"7452d789-b8dd-4ccc-b528-84ece1143cf9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5"
Jul 17 09:20:02.323561 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.323498    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7452d789-b8dd-4ccc-b528-84ece1143cf9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9pmr5\" (UID: \"7452d789-b8dd-4ccc-b528-84ece1143cf9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5"
Jul 17 09:20:02.424278 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424183    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37ac236b-cf30-427e-b6a9-0e1ee305a56e-ca-trust-extracted\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.424278 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424233    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7452d789-b8dd-4ccc-b528-84ece1143cf9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9pmr5\" (UID: \"7452d789-b8dd-4ccc-b528-84ece1143cf9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5"
Jul 17 09:20:02.424278 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424264    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37ac236b-cf30-427e-b6a9-0e1ee305a56e-image-registry-private-configuration\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.424592 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424308    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7452d789-b8dd-4ccc-b528-84ece1143cf9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9pmr5\" (UID: \"7452d789-b8dd-4ccc-b528-84ece1143cf9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5"
Jul 17 09:20:02.424592 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424331    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37ac236b-cf30-427e-b6a9-0e1ee305a56e-trusted-ca\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.424592 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424356    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37ac236b-cf30-427e-b6a9-0e1ee305a56e-registry-certificates\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.424592 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424380    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbt2m\" (UniqueName: \"kubernetes.io/projected/37ac236b-cf30-427e-b6a9-0e1ee305a56e-kube-api-access-fbt2m\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.424592 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424406    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47bv2\" (UniqueName: \"kubernetes.io/projected/9b64b23e-d02b-4b7b-9abb-edb2afe4b0a8-kube-api-access-47bv2\") pod \"downloads-5bc67dcd46-5fnqc\" (UID: \"9b64b23e-d02b-4b7b-9abb-edb2afe4b0a8\") " pod="openshift-console/downloads-5bc67dcd46-5fnqc"
Jul 17 09:20:02.424592 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424520    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37ac236b-cf30-427e-b6a9-0e1ee305a56e-registry-tls\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.424839 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424597    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37ac236b-cf30-427e-b6a9-0e1ee305a56e-bound-sa-token\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.424839 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424635    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37ac236b-cf30-427e-b6a9-0e1ee305a56e-installation-pull-secrets\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.424930 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.424911    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7452d789-b8dd-4ccc-b528-84ece1143cf9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-9pmr5\" (UID: \"7452d789-b8dd-4ccc-b528-84ece1143cf9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5"
Jul 17 09:20:02.426705 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.426682    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7452d789-b8dd-4ccc-b528-84ece1143cf9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-9pmr5\" (UID: \"7452d789-b8dd-4ccc-b528-84ece1143cf9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5"
Jul 17 09:20:02.477747 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.477722    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5"
Jul 17 09:20:02.526815 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.525868    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37ac236b-cf30-427e-b6a9-0e1ee305a56e-bound-sa-token\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.526815 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.525919    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37ac236b-cf30-427e-b6a9-0e1ee305a56e-installation-pull-secrets\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.526815 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.526008    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37ac236b-cf30-427e-b6a9-0e1ee305a56e-ca-trust-extracted\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.526815 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.526056    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37ac236b-cf30-427e-b6a9-0e1ee305a56e-image-registry-private-configuration\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.526815 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.526099    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37ac236b-cf30-427e-b6a9-0e1ee305a56e-trusted-ca\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.526815 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.526124    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37ac236b-cf30-427e-b6a9-0e1ee305a56e-registry-certificates\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.526815 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.526146    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbt2m\" (UniqueName: \"kubernetes.io/projected/37ac236b-cf30-427e-b6a9-0e1ee305a56e-kube-api-access-fbt2m\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.526815 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.526174    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47bv2\" (UniqueName: \"kubernetes.io/projected/9b64b23e-d02b-4b7b-9abb-edb2afe4b0a8-kube-api-access-47bv2\") pod \"downloads-5bc67dcd46-5fnqc\" (UID: \"9b64b23e-d02b-4b7b-9abb-edb2afe4b0a8\") " pod="openshift-console/downloads-5bc67dcd46-5fnqc"
Jul 17 09:20:02.526815 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.526211    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37ac236b-cf30-427e-b6a9-0e1ee305a56e-registry-tls\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.527358 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.527165    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37ac236b-cf30-427e-b6a9-0e1ee305a56e-ca-trust-extracted\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.527613 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.527589    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37ac236b-cf30-427e-b6a9-0e1ee305a56e-registry-certificates\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.528412 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.528383    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37ac236b-cf30-427e-b6a9-0e1ee305a56e-trusted-ca\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.529392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.529367    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37ac236b-cf30-427e-b6a9-0e1ee305a56e-installation-pull-secrets\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.529862 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.529839    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37ac236b-cf30-427e-b6a9-0e1ee305a56e-registry-tls\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.529939 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.529843    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37ac236b-cf30-427e-b6a9-0e1ee305a56e-image-registry-private-configuration\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.542516 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.542488    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47bv2\" (UniqueName: \"kubernetes.io/projected/9b64b23e-d02b-4b7b-9abb-edb2afe4b0a8-kube-api-access-47bv2\") pod \"downloads-5bc67dcd46-5fnqc\" (UID: \"9b64b23e-d02b-4b7b-9abb-edb2afe4b0a8\") " pod="openshift-console/downloads-5bc67dcd46-5fnqc"
Jul 17 09:20:02.543291 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.543270    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbt2m\" (UniqueName: \"kubernetes.io/projected/37ac236b-cf30-427e-b6a9-0e1ee305a56e-kube-api-access-fbt2m\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.544862 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.544841    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37ac236b-cf30-427e-b6a9-0e1ee305a56e-bound-sa-token\") pod \"image-registry-79b9779f5d-fzv2c\" (UID: \"37ac236b-cf30-427e-b6a9-0e1ee305a56e\") " pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.585891 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.585587    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-5fnqc"
Jul 17 09:20:02.590641 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.590618    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:02.626609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.626577    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:20:02.630083 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.629596    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/da993dbb-9c00-47e6-a410-8c8692e4644c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jprcn\" (UID: \"da993dbb-9c00-47e6-a410-8c8692e4644c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:20:02.656255 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.649589    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5"]
Jul 17 09:20:02.656334 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:20:02.656243    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7452d789_b8dd_4ccc_b528_84ece1143cf9.slice/crio-d1e1ede910d7c066b31098ce3c05deee76a20b09e82fbc092b9dbeec7dd7d311 WatchSource:0}: Error finding container d1e1ede910d7c066b31098ce3c05deee76a20b09e82fbc092b9dbeec7dd7d311: Status 404 returned error can't find the container with id d1e1ede910d7c066b31098ce3c05deee76a20b09e82fbc092b9dbeec7dd7d311
Jul 17 09:20:02.728880 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.728842    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-5fnqc"]
Jul 17 09:20:02.732947 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:20:02.732919    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b64b23e_d02b_4b7b_9abb_edb2afe4b0a8.slice/crio-1268a20505d44e031d8aafeb32179558ce3b08a4de807adb4755d717399962c2 WatchSource:0}: Error finding container 1268a20505d44e031d8aafeb32179558ce3b08a4de807adb4755d717399962c2: Status 404 returned error can't find the container with id 1268a20505d44e031d8aafeb32179558ce3b08a4de807adb4755d717399962c2
Jul 17 09:20:02.751949 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.751925    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79b9779f5d-fzv2c"]
Jul 17 09:20:02.754593 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:20:02.754566    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ac236b_cf30_427e_b6a9_0e1ee305a56e.slice/crio-9139e494def390b6859c6644421ae1a4e6d75de8bed8a37150a8f7fbecb9ab97 WatchSource:0}: Error finding container 9139e494def390b6859c6644421ae1a4e6d75de8bed8a37150a8f7fbecb9ab97: Status 404 returned error can't find the container with id 9139e494def390b6859c6644421ae1a4e6d75de8bed8a37150a8f7fbecb9ab97
Jul 17 09:20:02.902721 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.902694    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pmw5q\""
Jul 17 09:20:02.910056 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:02.910033    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"
Jul 17 09:20:03.045027 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:03.044994    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn"]
Jul 17 09:20:03.048621 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:20:03.048586    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda993dbb_9c00_47e6_a410_8c8692e4644c.slice/crio-e9fb005f95f28769bad855cbe33b3fe21076121d3cdd16ed4f72b19e24fa3af9 WatchSource:0}: Error finding container e9fb005f95f28769bad855cbe33b3fe21076121d3cdd16ed4f72b19e24fa3af9: Status 404 returned error can't find the container with id e9fb005f95f28769bad855cbe33b3fe21076121d3cdd16ed4f72b19e24fa3af9
Jul 17 09:20:03.090591 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:03.090552    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c" event={"ID":"37ac236b-cf30-427e-b6a9-0e1ee305a56e","Type":"ContainerStarted","Data":"c63137a2f71331b4a9c604c1a871870c0bdb4e2d742cd12d4b6b419c91ef4e90"}
Jul 17 09:20:03.090784 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:03.090616    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c" event={"ID":"37ac236b-cf30-427e-b6a9-0e1ee305a56e","Type":"ContainerStarted","Data":"9139e494def390b6859c6644421ae1a4e6d75de8bed8a37150a8f7fbecb9ab97"}
Jul 17 09:20:03.090784 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:03.090670    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:03.091930 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:03.091907    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5" event={"ID":"7452d789-b8dd-4ccc-b528-84ece1143cf9","Type":"ContainerStarted","Data":"d1e1ede910d7c066b31098ce3c05deee76a20b09e82fbc092b9dbeec7dd7d311"}
Jul 17 09:20:03.092940 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:03.092919    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn" event={"ID":"da993dbb-9c00-47e6-a410-8c8692e4644c","Type":"ContainerStarted","Data":"e9fb005f95f28769bad855cbe33b3fe21076121d3cdd16ed4f72b19e24fa3af9"}
Jul 17 09:20:03.093995 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:03.093976    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-5fnqc" event={"ID":"9b64b23e-d02b-4b7b-9abb-edb2afe4b0a8","Type":"ContainerStarted","Data":"1268a20505d44e031d8aafeb32179558ce3b08a4de807adb4755d717399962c2"}
Jul 17 09:20:03.119475 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:03.119422    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c" podStartSLOduration=1.119404532 podStartE2EDuration="1.119404532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:03.118832487 +0000 UTC m=+87.183653212" watchObservedRunningTime="2026-07-17 09:20:03.119404532 +0000 UTC m=+87.184225249"
Jul 17 09:20:06.108699 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.108653    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5" event={"ID":"7452d789-b8dd-4ccc-b528-84ece1143cf9","Type":"ContainerStarted","Data":"9de4586f9dc301f2da501c437eba3646d8e587a96cadc4d7cd2ba683f382db04"}
Jul 17 09:20:06.110294 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.110269    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn" event={"ID":"da993dbb-9c00-47e6-a410-8c8692e4644c","Type":"ContainerStarted","Data":"2b77e8ab02a3e707d276463b6beb3140ebf3c241ed9f61a4e2803fc96381d799"}
Jul 17 09:20:06.158909 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.158876    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:20:06.161774 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.161744    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-48h52\" (UID: \"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb\") " pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:20:06.213852 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.213796    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-9pmr5" podStartSLOduration=1.434007115 podStartE2EDuration="4.213775495s" totalImagesPullingTime="2.77976838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:06.157628971 +0000 UTC m=+90.222449694" watchObservedRunningTime="2026-07-17 09:20:06.213775495 +0000 UTC m=+90.278596218"
Jul 17 09:20:06.214129 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.214106    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jprcn" podStartSLOduration=33.638294051 podStartE2EDuration="36.214100349s" totalImagesPullingTime="2.575806298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:06.206612438 +0000 UTC m=+90.271433163" watchObservedRunningTime="2026-07-17 09:20:06.214100349 +0000 UTC m=+90.278921071"
Jul 17 09:20:06.415595 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.415510    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cb5vz\""
Jul 17 09:20:06.422017 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.421987    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-48h52"
Jul 17 09:20:06.561477 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.561440    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg"]
Jul 17 09:20:06.564244 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.564214    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg"
Jul 17 09:20:06.567155 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.567133    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 17 09:20:06.567406 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.567390    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-2sch4\""
Jul 17 09:20:06.576263 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.576239    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg"]
Jul 17 09:20:06.584492 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.584452    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-48h52"]
Jul 17 09:20:06.588062 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:20:06.588021    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0b1c0c_9028_4f4f_9ea7_f2da0858bceb.slice/crio-ffb625dfe882ded7a5ffccd044b2fcf8b07a1c2d02c27bdc5f791f4b315de72d WatchSource:0}: Error finding container ffb625dfe882ded7a5ffccd044b2fcf8b07a1c2d02c27bdc5f791f4b315de72d: Status 404 returned error can't find the container with id ffb625dfe882ded7a5ffccd044b2fcf8b07a1c2d02c27bdc5f791f4b315de72d
Jul 17 09:20:06.664327 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.664280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f0ca7342-a808-4f5a-a119-9efb356b05f8-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-l8zfg\" (UID: \"f0ca7342-a808-4f5a-a119-9efb356b05f8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg"
Jul 17 09:20:06.764790 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.764747    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f0ca7342-a808-4f5a-a119-9efb356b05f8-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-l8zfg\" (UID: \"f0ca7342-a808-4f5a-a119-9efb356b05f8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg"
Jul 17 09:20:06.767455 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.767429    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f0ca7342-a808-4f5a-a119-9efb356b05f8-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-l8zfg\" (UID: \"f0ca7342-a808-4f5a-a119-9efb356b05f8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg"
Jul 17 09:20:06.876235 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:06.876199    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg"
Jul 17 09:20:07.041894 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:07.041814    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg"]
Jul 17 09:20:07.119640 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:07.119601    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-48h52" event={"ID":"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb","Type":"ContainerStarted","Data":"ffb625dfe882ded7a5ffccd044b2fcf8b07a1c2d02c27bdc5f791f4b315de72d"}
Jul 17 09:20:07.122502 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:07.122413    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg" event={"ID":"f0ca7342-a808-4f5a-a119-9efb356b05f8","Type":"ContainerStarted","Data":"f5d8bdf7a019dad9aa8e35e9c4367d006e0647e59d990b945d71c1fd5d80a3c7"}
Jul 17 09:20:09.133329 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:09.133290    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg" event={"ID":"f0ca7342-a808-4f5a-a119-9efb356b05f8","Type":"ContainerStarted","Data":"5251ecd0c0947e010dada16d1211608a3c50db062551f2b7d0328a46b954f460"}
Jul 17 09:20:09.134361 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:09.134333    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg"
Jul 17 09:20:09.140039 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:09.140013    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg"
Jul 17 09:20:09.204367 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:09.204294    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-l8zfg" podStartSLOduration=1.6877974839999998 podStartE2EDuration="3.204273156s" totalImagesPullingTime="1.516475672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:09.173418033 +0000 UTC m=+93.238238756" watchObservedRunningTime="2026-07-17 09:20:09.204273156 +0000 UTC m=+93.269093879"
Jul 17 09:20:12.155209 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:12.155182    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:20:13.156511 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:13.156447    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-48h52" event={"ID":"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb","Type":"ContainerStarted","Data":"61993921c4bb03f739c75c88d38c14f26bb873d02d2af0e4f883edcfc1cb4a11"}
Jul 17 09:20:15.030709 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:15.030680    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jlhnc"
Jul 17 09:20:16.343233 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:16.343188    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:20:16.345817 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:16.345791    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4db5b93e-8ebb-4d86-b45a-0bee6a6890f8-metrics-tls\") pod \"dns-default-pvp8l\" (UID: \"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8\") " pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:20:16.561347 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:16.561316    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4mh5z\""
Jul 17 09:20:16.568223 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:16.568199    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:20:17.729635 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.729595    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"]
Jul 17 09:20:17.753737 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.753683    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.757929 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.757893    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.758110 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.757963    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.758110 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.758011    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.758110 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.758043    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.758110 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.758090    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.758367 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.758175    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh99p\" (UniqueName: \"kubernetes.io/projected/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-api-access-vh99p\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.760345 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.760319    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"]
Jul 17 09:20:17.760479 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.760412    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6sp4v"]
Jul 17 09:20:17.763927 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.763890    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-g5d9z\""
Jul 17 09:20:17.764248 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.764209    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 17 09:20:17.764649 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.764283    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 09:20:17.764903 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.764880    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 17 09:20:17.766550 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.766527    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 17 09:20:17.788290 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.788260    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.792864 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.792842    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 09:20:17.792864 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.792854    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 09:20:17.793050 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.792845    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rq7jd\""
Jul 17 09:20:17.793183 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.793166    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 09:20:17.858536 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858500    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh99p\" (UniqueName: \"kubernetes.io/projected/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-api-access-vh99p\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.858711 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858543    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-wtmp\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.858711 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858572    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.858711 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858599    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25dfceba-0dab-4034-8441-364ad8a10ba1-sys\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.858711 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858649    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-textfile\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.858711 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858696    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.858979 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858786    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.858979 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858819    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25dfceba-0dab-4034-8441-364ad8a10ba1-metrics-client-ca\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.858979 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858872    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.858979 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858916    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.858979 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858949    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.859227 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.858979    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25dfceba-0dab-4034-8441-364ad8a10ba1-root\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.859227 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.859027    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-tls\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.859227 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.859128    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftv9c\" (UniqueName: \"kubernetes.io/projected/25dfceba-0dab-4034-8441-364ad8a10ba1-kube-api-access-ftv9c\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.859227 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.859163    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.859429 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.859335    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.859699 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.859660    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.859837 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.859761    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.861840 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.861817    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.861948 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.861859    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.870890 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.870867    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh99p\" (UniqueName: \"kubernetes.io/projected/10c073d2-4eee-45f2-9f5e-e15b7afc16d5-kube-api-access-vh99p\") pod \"kube-state-metrics-56c5594674-zsf4n\" (UID: \"10c073d2-4eee-45f2-9f5e-e15b7afc16d5\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:17.959773 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.959741    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25dfceba-0dab-4034-8441-364ad8a10ba1-metrics-client-ca\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.959984 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.959820    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25dfceba-0dab-4034-8441-364ad8a10ba1-root\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.959984 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.959852    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-tls\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.959984 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.959880    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftv9c\" (UniqueName: \"kubernetes.io/projected/25dfceba-0dab-4034-8441-364ad8a10ba1-kube-api-access-ftv9c\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.959984 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.959905    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-wtmp\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.959984 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.959940    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/25dfceba-0dab-4034-8441-364ad8a10ba1-root\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.960190 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.960020    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-wtmp\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.960190 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.960125    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.960190 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.960156    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25dfceba-0dab-4034-8441-364ad8a10ba1-sys\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.960190 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.960178    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-textfile\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.960326 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.960225    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.960392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.960372    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/25dfceba-0dab-4034-8441-364ad8a10ba1-metrics-client-ca\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.960453 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.960438    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/25dfceba-0dab-4034-8441-364ad8a10ba1-sys\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.960599 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.960575    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-textfile\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.960757 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.960733    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.963074 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.963054    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.963291 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.963262    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/25dfceba-0dab-4034-8441-364ad8a10ba1-node-exporter-tls\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:17.989531 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:17.989449    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftv9c\" (UniqueName: \"kubernetes.io/projected/25dfceba-0dab-4034-8441-364ad8a10ba1-kube-api-access-ftv9c\") pod \"node-exporter-6sp4v\" (UID: \"25dfceba-0dab-4034-8441-364ad8a10ba1\") " pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:18.067679 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:18.067641    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"
Jul 17 09:20:18.098908 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:18.098883    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6sp4v"
Jul 17 09:20:21.622370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.622333    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-59d865899b-9pxjh"]
Jul 17 09:20:21.634673 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.634648    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.638314 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.638242    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 17 09:20:21.638483 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.638326    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-t4qd2\""
Jul 17 09:20:21.638483 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.638353    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 17 09:20:21.638711 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.638692    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d8ujdkmp549ca\""
Jul 17 09:20:21.639416 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.639357    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 17 09:20:21.639416 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.639389    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 17 09:20:21.639596 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.639389    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 17 09:20:21.649122 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.649101    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59d865899b-9pxjh"]
Jul 17 09:20:21.693928 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.693895    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.694097 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.693939    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.694097 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.693962    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.694097 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.694035    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb566\" (UniqueName: \"kubernetes.io/projected/8cb71f1a-71a5-4302-8751-f2ec2196b256-kube-api-access-hb566\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.694325 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.694114    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-tls\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.694325 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.694175    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.694325 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.694200    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cb71f1a-71a5-4302-8751-f2ec2196b256-metrics-client-ca\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.694325 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.694216    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-grpc-tls\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.795587 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.795550    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.795744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.795601    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.795744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.795630    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.795744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.795664    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb566\" (UniqueName: \"kubernetes.io/projected/8cb71f1a-71a5-4302-8751-f2ec2196b256-kube-api-access-hb566\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.795744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.795708    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-tls\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.795991 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.795756    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.795991 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.795927    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cb71f1a-71a5-4302-8751-f2ec2196b256-metrics-client-ca\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.795991 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.795966    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-grpc-tls\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.796814 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.796763    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cb71f1a-71a5-4302-8751-f2ec2196b256-metrics-client-ca\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.798638 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.798611    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.798794 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.798773    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.799177 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.799150    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.799336 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.799315    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-grpc-tls\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.799403 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.799335    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.799538 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.799513    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/8cb71f1a-71a5-4302-8751-f2ec2196b256-secret-thanos-querier-tls\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.808563 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.808536    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb566\" (UniqueName: \"kubernetes.io/projected/8cb71f1a-71a5-4302-8751-f2ec2196b256-kube-api-access-hb566\") pod \"thanos-querier-59d865899b-9pxjh\" (UID: \"8cb71f1a-71a5-4302-8751-f2ec2196b256\") " pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.945587 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.945405    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:21.947232 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.947196    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"]
Jul 17 09:20:21.959752 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.959513    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"
Jul 17 09:20:21.961248 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.961222    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"]
Jul 17 09:20:21.962585 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.962563    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-7b76k\""
Jul 17 09:20:21.962854 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.962839    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 17 09:20:21.997851 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.997816    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-nn2z7\" (UID: \"6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"
Jul 17 09:20:21.997994 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:21.997874    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhzwj\" (UniqueName: \"kubernetes.io/projected/6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7-kube-api-access-lhzwj\") pod \"monitoring-plugin-7b7b7bc8bf-nn2z7\" (UID: \"6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"
Jul 17 09:20:22.098862 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:22.098831    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-nn2z7\" (UID: \"6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"
Jul 17 09:20:22.098862 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:22.098874    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhzwj\" (UniqueName: \"kubernetes.io/projected/6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7-kube-api-access-lhzwj\") pod \"monitoring-plugin-7b7b7bc8bf-nn2z7\" (UID: \"6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"
Jul 17 09:20:22.101308 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:22.101272    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-nn2z7\" (UID: \"6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"
Jul 17 09:20:22.111309 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:22.111283    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhzwj\" (UniqueName: \"kubernetes.io/projected/6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7-kube-api-access-lhzwj\") pod \"monitoring-plugin-7b7b7bc8bf-nn2z7\" (UID: \"6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"
Jul 17 09:20:22.272057 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:22.271966    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"
Jul 17 09:20:22.752136 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:20:22.752109    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25dfceba_0dab_4034_8441_364ad8a10ba1.slice/crio-4ff92bc7befce500865954633b87380d254248a412f02ef11b7d6c715a27678c WatchSource:0}: Error finding container 4ff92bc7befce500865954633b87380d254248a412f02ef11b7d6c715a27678c: Status 404 returned error can't find the container with id 4ff92bc7befce500865954633b87380d254248a412f02ef11b7d6c715a27678c
Jul 17 09:20:22.958220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:22.958187    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-zsf4n"]
Jul 17 09:20:22.960380 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:20:22.960352    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c073d2_4eee_45f2_9f5e_e15b7afc16d5.slice/crio-339ec3eae5e0689accdd8a43922de4f16e417e571fe760a5c88a8ea48fd4df1f WatchSource:0}: Error finding container 339ec3eae5e0689accdd8a43922de4f16e417e571fe760a5c88a8ea48fd4df1f: Status 404 returned error can't find the container with id 339ec3eae5e0689accdd8a43922de4f16e417e571fe760a5c88a8ea48fd4df1f
Jul 17 09:20:23.162891 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.162807    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"]
Jul 17 09:20:23.168213 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.168016    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pvp8l"]
Jul 17 09:20:23.180577 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.178840    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-59d865899b-9pxjh"]
Jul 17 09:20:23.180577 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:20:23.179662    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db5b93e_8ebb_4d86_b45a_0bee6a6890f8.slice/crio-b4ec47a64a4bb15aeac596453a78b1a6ccb8565ba938ad54293741ee2251e125 WatchSource:0}: Error finding container b4ec47a64a4bb15aeac596453a78b1a6ccb8565ba938ad54293741ee2251e125: Status 404 returned error can't find the container with id b4ec47a64a4bb15aeac596453a78b1a6ccb8565ba938ad54293741ee2251e125
Jul 17 09:20:23.183989 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:20:23.183968    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb71f1a_71a5_4302_8751_f2ec2196b256.slice/crio-c255a289cfb1b04d3332e017c5ae9beb2d27c5433296dc8604f0dfa2e48752e4 WatchSource:0}: Error finding container c255a289cfb1b04d3332e017c5ae9beb2d27c5433296dc8604f0dfa2e48752e4: Status 404 returned error can't find the container with id c255a289cfb1b04d3332e017c5ae9beb2d27c5433296dc8604f0dfa2e48752e4
Jul 17 09:20:23.197032 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.196928    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7" event={"ID":"6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7","Type":"ContainerStarted","Data":"08601a110fe992b00a0d7bf37bcdbed00c747e6b268fe9bae3cb63a850c0d203"}
Jul 17 09:20:23.200133 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.200101    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh" event={"ID":"8cb71f1a-71a5-4302-8751-f2ec2196b256","Type":"ContainerStarted","Data":"c255a289cfb1b04d3332e017c5ae9beb2d27c5433296dc8604f0dfa2e48752e4"}
Jul 17 09:20:23.204336 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.201722    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvp8l" event={"ID":"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8","Type":"ContainerStarted","Data":"b4ec47a64a4bb15aeac596453a78b1a6ccb8565ba938ad54293741ee2251e125"}
Jul 17 09:20:23.204659 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.204601    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-48h52" event={"ID":"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb","Type":"ContainerStarted","Data":"a77fef801c91d6afb45d890ea5f4578ac393ad320f703cfcc4f7f49ed279c64d"}
Jul 17 09:20:23.206883 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.206851    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-5fnqc" event={"ID":"9b64b23e-d02b-4b7b-9abb-edb2afe4b0a8","Type":"ContainerStarted","Data":"6c59623dd67846aab8329061ce69b2b5a60824306f0f8ccc349973890940aebc"}
Jul 17 09:20:23.207749 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.207211    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-5fnqc"
Jul 17 09:20:23.209048 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.208976    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n" event={"ID":"10c073d2-4eee-45f2-9f5e-e15b7afc16d5","Type":"ContainerStarted","Data":"339ec3eae5e0689accdd8a43922de4f16e417e571fe760a5c88a8ea48fd4df1f"}
Jul 17 09:20:23.210413 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.210389    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sp4v" event={"ID":"25dfceba-0dab-4034-8441-364ad8a10ba1","Type":"ContainerStarted","Data":"4ff92bc7befce500865954633b87380d254248a412f02ef11b7d6c715a27678c"}
Jul 17 09:20:23.218761 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.218734    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-5fnqc"
Jul 17 09:20:23.231821 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:23.230607    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-5fnqc" podStartSLOduration=1.217797624 podStartE2EDuration="21.230586274s" totalImagesPullingTime="20.01278865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:23.229521076 +0000 UTC m=+107.294341798" watchObservedRunningTime="2026-07-17 09:20:23.230586274 +0000 UTC m=+107.295407003"
Jul 17 09:20:24.107239 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:24.106306    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79b9779f5d-fzv2c"
Jul 17 09:20:25.229102 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:25.229037    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sp4v" event={"ID":"25dfceba-0dab-4034-8441-364ad8a10ba1","Type":"ContainerStarted","Data":"f119b25508e8f947daba81eea37bb062d50750006c3c6f86f2b14c1d08ca20d3"}
Jul 17 09:20:27.168570 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:27.168511    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6d596888c-k5kh6" podUID="394741d2-bf2f-4627-801d-8721cf6071c0" containerName="registry" containerID="cri-o://088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1" gracePeriod=30
Jul 17 09:20:27.245650 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:27.245613    2571 generic.go:356] "Generic (PLEG): container finished" podID="25dfceba-0dab-4034-8441-364ad8a10ba1" containerID="f119b25508e8f947daba81eea37bb062d50750006c3c6f86f2b14c1d08ca20d3" exitCode=0
Jul 17 09:20:27.245794 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:27.245695    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sp4v" event={"ID":"25dfceba-0dab-4034-8441-364ad8a10ba1","Type":"ContainerDied","Data":"f119b25508e8f947daba81eea37bb062d50750006c3c6f86f2b14c1d08ca20d3"}
Jul 17 09:20:30.865810 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:30.864808    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:20:31.000595 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.000303    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cq749\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-kube-api-access-cq749\") pod \"394741d2-bf2f-4627-801d-8721cf6071c0\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") "
Jul 17 09:20:31.000595 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.000349    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-image-registry-private-configuration\") pod \"394741d2-bf2f-4627-801d-8721cf6071c0\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") "
Jul 17 09:20:31.000595 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.000388    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-bound-sa-token\") pod \"394741d2-bf2f-4627-801d-8721cf6071c0\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") "
Jul 17 09:20:31.000595 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.000425    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-installation-pull-secrets\") pod \"394741d2-bf2f-4627-801d-8721cf6071c0\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") "
Jul 17 09:20:31.001620 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.000945    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/394741d2-bf2f-4627-801d-8721cf6071c0-ca-trust-extracted\") pod \"394741d2-bf2f-4627-801d-8721cf6071c0\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") "
Jul 17 09:20:31.001620 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.001001    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-registry-certificates\") pod \"394741d2-bf2f-4627-801d-8721cf6071c0\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") "
Jul 17 09:20:31.001620 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.001073    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-trusted-ca\") pod \"394741d2-bf2f-4627-801d-8721cf6071c0\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") "
Jul 17 09:20:31.001620 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.001101    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls\") pod \"394741d2-bf2f-4627-801d-8721cf6071c0\" (UID: \"394741d2-bf2f-4627-801d-8721cf6071c0\") "
Jul 17 09:20:31.006231 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.006173    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "394741d2-bf2f-4627-801d-8721cf6071c0" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:20:31.010974 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.006411    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "394741d2-bf2f-4627-801d-8721cf6071c0" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:20:31.010974 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.009915    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "394741d2-bf2f-4627-801d-8721cf6071c0" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:20:31.021345 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.020894    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/394741d2-bf2f-4627-801d-8721cf6071c0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "394741d2-bf2f-4627-801d-8721cf6071c0" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:20:31.033140 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.033086    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "394741d2-bf2f-4627-801d-8721cf6071c0" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:20:31.033543 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.033510    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "394741d2-bf2f-4627-801d-8721cf6071c0" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:20:31.034031 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.033986    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-kube-api-access-cq749" (OuterVolumeSpecName: "kube-api-access-cq749") pod "394741d2-bf2f-4627-801d-8721cf6071c0" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0"). InnerVolumeSpecName "kube-api-access-cq749". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:20:31.037501 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.036928    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "394741d2-bf2f-4627-801d-8721cf6071c0" (UID: "394741d2-bf2f-4627-801d-8721cf6071c0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:20:31.102687 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.102548    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-trusted-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:20:31.102687 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.102576    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-registry-tls\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:20:31.102687 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.102591    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cq749\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-kube-api-access-cq749\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:20:31.102687 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.102605    2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-image-registry-private-configuration\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:20:31.102687 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.102619    2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/394741d2-bf2f-4627-801d-8721cf6071c0-bound-sa-token\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:20:31.102687 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.102633    2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/394741d2-bf2f-4627-801d-8721cf6071c0-installation-pull-secrets\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:20:31.102687 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.102646    2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/394741d2-bf2f-4627-801d-8721cf6071c0-ca-trust-extracted\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:20:31.102687 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.102660    2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/394741d2-bf2f-4627-801d-8721cf6071c0-registry-certificates\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:20:31.272279 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.272239    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n" event={"ID":"10c073d2-4eee-45f2-9f5e-e15b7afc16d5","Type":"ContainerStarted","Data":"51f8b867f42b5eabc2861518b9551ade4206736bc06ddb60c959b4414d988fe5"}
Jul 17 09:20:31.272444 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.272351    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n" event={"ID":"10c073d2-4eee-45f2-9f5e-e15b7afc16d5","Type":"ContainerStarted","Data":"89cfe6940d4a948727dfd8e697d49494a6af53c8cf4bd414e6f7b32861b7f47f"}
Jul 17 09:20:31.272444 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.272383    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n" event={"ID":"10c073d2-4eee-45f2-9f5e-e15b7afc16d5","Type":"ContainerStarted","Data":"6cba01693349bf3de3a08f86830f1240818a09a1019e084b997d124046eda59b"}
Jul 17 09:20:31.274662 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.274634    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sp4v" event={"ID":"25dfceba-0dab-4034-8441-364ad8a10ba1","Type":"ContainerStarted","Data":"66b78c651cc571e0a1ba8ddc957ad0853b9333993ab5f3628c65122d11149f42"}
Jul 17 09:20:31.274848 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.274828    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sp4v" event={"ID":"25dfceba-0dab-4034-8441-364ad8a10ba1","Type":"ContainerStarted","Data":"67fb4c2ea0d176ff944947dd681789e44b0c612d6ddc07065b948ece45833038"}
Jul 17 09:20:31.276756 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.276732    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7" event={"ID":"6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7","Type":"ContainerStarted","Data":"8cb2195c43be6b56cac19fb12d3fd7231d4a7d850660fc2972f660d25cf79e95"}
Jul 17 09:20:31.276988 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.276971    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"
Jul 17 09:20:31.278705 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.278681    2571 generic.go:356] "Generic (PLEG): container finished" podID="394741d2-bf2f-4627-801d-8721cf6071c0" containerID="088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1" exitCode=0
Jul 17 09:20:31.278780 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.278766    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d596888c-k5kh6" event={"ID":"394741d2-bf2f-4627-801d-8721cf6071c0","Type":"ContainerDied","Data":"088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1"}
Jul 17 09:20:31.278819 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.278797    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d596888c-k5kh6" event={"ID":"394741d2-bf2f-4627-801d-8721cf6071c0","Type":"ContainerDied","Data":"715a562ef6a8c5341eea7195e72f31488cd0d25371cbd38555dcc7c1006c3a9a"}
Jul 17 09:20:31.278852 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.278838    2571 scope.go:117] "RemoveContainer" containerID="088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1"
Jul 17 09:20:31.278988 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.278975    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d596888c-k5kh6"
Jul 17 09:20:31.284632 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.284576    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh" event={"ID":"8cb71f1a-71a5-4302-8751-f2ec2196b256","Type":"ContainerStarted","Data":"8af3fb3b13c84d78a43cd52c72fb808e90f4c9c86cb665d974d3ddd7e1202138"}
Jul 17 09:20:31.285756 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.284809    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh" event={"ID":"8cb71f1a-71a5-4302-8751-f2ec2196b256","Type":"ContainerStarted","Data":"047346e2b3b216d3715e5eb6272602c2677aeab6fee007b2cfc7ec802d80ca0e"}
Jul 17 09:20:31.286098 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.286081    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh" event={"ID":"8cb71f1a-71a5-4302-8751-f2ec2196b256","Type":"ContainerStarted","Data":"5c15d1e84c4fe97a015e81ee1231cb2d8bd1737a30d7411eae7795f2123d7c1d"}
Jul 17 09:20:31.288586 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.288563    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvp8l" event={"ID":"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8","Type":"ContainerStarted","Data":"59bb0a4ee8603119cf179cbbad49b466034b41bd8c1822174a6ce469db03e90e"}
Jul 17 09:20:31.288689 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.288607    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:20:31.288689 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.288649    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7"
Jul 17 09:20:31.288689 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.288666    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvp8l" event={"ID":"4db5b93e-8ebb-4d86-b45a-0bee6a6890f8","Type":"ContainerStarted","Data":"81b74728ecb97e4f430463b3a453b3a6b2335fb372aece1860d5272b5655315d"}
Jul 17 09:20:31.290533 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.289697    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-48h52" event={"ID":"bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb","Type":"ContainerStarted","Data":"e16081d6da3dac40a3ef88e1d57b7716240b16be6d76922a3700161cbcffadb3"}
Jul 17 09:20:31.292224 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.292204    2571 scope.go:117] "RemoveContainer" containerID="088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1"
Jul 17 09:20:31.292526 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:20:31.292505    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1\": container with ID starting with 088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1 not found: ID does not exist" containerID="088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1"
Jul 17 09:20:31.292620 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.292531    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1"} err="failed to get container status \"088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1\": rpc error: code = NotFound desc = could not find container \"088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1\": container with ID starting with 088580aab588b88dd746a78cd6a7fae116c17eed9178e5f7d08b9e98442797f1 not found: ID does not exist"
Jul 17 09:20:31.296054 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.296016    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-zsf4n" podStartSLOduration=6.566904701 podStartE2EDuration="14.296004304s" totalImagesPullingTime="7.729099603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:31.294981222 +0000 UTC m=+115.359801947" watchObservedRunningTime="2026-07-17 09:20:31.296004304 +0000 UTC m=+115.360825025"
Jul 17 09:20:31.313458 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.313419    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-nn2z7" podStartSLOduration=2.791329434 podStartE2EDuration="10.313406082s" totalImagesPullingTime="7.522076648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:31.312455357 +0000 UTC m=+115.377276079" watchObservedRunningTime="2026-07-17 09:20:31.313406082 +0000 UTC m=+115.378226782"
Jul 17 09:20:31.335075 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.332918    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-48h52" podStartSLOduration=33.381563544 podStartE2EDuration="57.332901949s" totalImagesPullingTime="23.951338405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:19:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:31.332241554 +0000 UTC m=+115.397062280" watchObservedRunningTime="2026-07-17 09:20:31.332901949 +0000 UTC m=+115.397722672"
Jul 17 09:20:31.348183 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.348157    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d596888c-k5kh6"]
Jul 17 09:20:31.352200 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.352173    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d596888c-k5kh6"]
Jul 17 09:20:31.370549 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.370490    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pvp8l" podStartSLOduration=71.861873913 podStartE2EDuration="1m19.370446794s" totalImagesPullingTime="7.508572881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:31.368986164 +0000 UTC m=+115.433806887" watchObservedRunningTime="2026-07-17 09:20:31.370446794 +0000 UTC m=+115.435267517"
Jul 17 09:20:31.389076 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.389023    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6sp4v" podStartSLOduration=12.680484553 podStartE2EDuration="14.389008351s" totalImagesPullingTime="1.579073747s" totalInitContainerRuntime="129.450051ms" isStatefulPod=true podCreationTimestamp="2026-07-17 09:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:31.387849756 +0000 UTC m=+115.452670480" watchObservedRunningTime="2026-07-17 09:20:31.389008351 +0000 UTC m=+115.453829072"
Jul 17 09:20:31.758939 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:31.758907    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="394741d2-bf2f-4627-801d-8721cf6071c0" path="/var/lib/kubelet/pods/394741d2-bf2f-4627-801d-8721cf6071c0/volumes"
Jul 17 09:20:33.306478 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:33.306429    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh" event={"ID":"8cb71f1a-71a5-4302-8751-f2ec2196b256","Type":"ContainerStarted","Data":"1848fdde1fca43bebc681a859fdea83110463f2e1e04eddaf2f527ce54f30885"}
Jul 17 09:20:33.307013 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:33.306525    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh" event={"ID":"8cb71f1a-71a5-4302-8751-f2ec2196b256","Type":"ContainerStarted","Data":"a41d964dbab9f7a1e945e754ec3fbdf96f55b813a205718bbcb83264fd53bcf5"}
Jul 17 09:20:33.307013 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:33.306552    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh" event={"ID":"8cb71f1a-71a5-4302-8751-f2ec2196b256","Type":"ContainerStarted","Data":"3c3ae31a9de33ab6dae15ba1d4ea5d330b1515a5dfaf94462f0012e35cddafa9"}
Jul 17 09:20:34.310497 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:34.310448    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:40.321226 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:40.321192    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh"
Jul 17 09:20:40.351018 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:40.350963    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-59d865899b-9pxjh" podStartSLOduration=10.630960598 podStartE2EDuration="19.350946435s" totalImagesPullingTime="8.719985837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:20:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:33.342866176 +0000 UTC m=+117.407686901" watchObservedRunningTime="2026-07-17 09:20:40.350946435 +0000 UTC m=+124.415767156"
Jul 17 09:20:41.296453 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:41.296423    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pvp8l"
Jul 17 09:20:47.242628 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:47.242587    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:20:47.244896 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:47.244874    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea-metrics-certs\") pod \"network-metrics-daemon-sttnl\" (UID: \"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea\") " pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:20:47.368309 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:47.368278    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t2lf7\""
Jul 17 09:20:47.376511 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:47.376493    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sttnl"
Jul 17 09:20:47.591077 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:47.589911    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sttnl"]
Jul 17 09:20:48.363509 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:48.363444    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sttnl" event={"ID":"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea","Type":"ContainerStarted","Data":"d60ef406e38f51e5a1ccac9728288ffddbfad5666cbc1abb4aec38db1e485e5e"}
Jul 17 09:20:49.368776 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:49.368743    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sttnl" event={"ID":"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea","Type":"ContainerStarted","Data":"e1214de8559d63ba728064afda17f51267b7cb624575dc2b99643516eacf187c"}
Jul 17 09:20:50.380542 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:50.380503    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sttnl" event={"ID":"ef692b5e-e50a-4ced-8e5d-20b14fd8f0ea","Type":"ContainerStarted","Data":"f9f2576582dfd7e14c878680ba7b717a2924abdd82c1078dd8153af0dada1ae2"}
Jul 17 09:20:50.410875 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:50.410816    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sttnl" podStartSLOduration=130.808088973 podStartE2EDuration="2m12.410799661s" totalImagesPullingTime="1.602710688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:20:50.410054779 +0000 UTC m=+134.474875502" watchObservedRunningTime="2026-07-17 09:20:50.410799661 +0000 UTC m=+134.475620384"
Jul 17 09:20:54.398543 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:54.398507    2571 generic.go:356] "Generic (PLEG): container finished" podID="96b39859-a448-4d17-aa25-35628ab46939" containerID="7d6c0f9aafba63768c9df528a2a7f14d0c53c31b90cdc007f2de19f08b2eaa65" exitCode=0
Jul 17 09:20:54.398543 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:54.398545    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-98q75" event={"ID":"96b39859-a448-4d17-aa25-35628ab46939","Type":"ContainerDied","Data":"7d6c0f9aafba63768c9df528a2a7f14d0c53c31b90cdc007f2de19f08b2eaa65"}
Jul 17 09:20:54.399070 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:54.398873    2571 scope.go:117] "RemoveContainer" containerID="7d6c0f9aafba63768c9df528a2a7f14d0c53c31b90cdc007f2de19f08b2eaa65"
Jul 17 09:20:55.404772 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:20:55.404735    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-98q75" event={"ID":"96b39859-a448-4d17-aa25-35628ab46939","Type":"ContainerStarted","Data":"3df2e79b19fecb6a0a4094f2d0b0c8a227b94afd5f388c895b00df28fe4b573b"}
Jul 17 09:22:57.880795 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:57.880763    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"]
Jul 17 09:22:57.881307 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:57.881055    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="394741d2-bf2f-4627-801d-8721cf6071c0" containerName="registry"
Jul 17 09:22:57.881307 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:57.881067    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="394741d2-bf2f-4627-801d-8721cf6071c0" containerName="registry"
Jul 17 09:22:57.881307 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:57.881128    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="394741d2-bf2f-4627-801d-8721cf6071c0" containerName="registry"
Jul 17 09:22:57.883080 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:57.883060    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:22:57.885759 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:57.885739    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 17 09:22:57.886680 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:57.886662    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 17 09:22:57.886680 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:57.886671    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-97w4q\""
Jul 17 09:22:57.895565 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:57.895512    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"]
Jul 17 09:22:58.000315 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.000278    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dxk8\" (UniqueName: \"kubernetes.io/projected/330d0380-4c84-4e0d-94d7-ef420bc9a155-kube-api-access-6dxk8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:22:58.000512 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.000330    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:22:58.000512 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.000396    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:22:58.101519 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.101450    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dxk8\" (UniqueName: \"kubernetes.io/projected/330d0380-4c84-4e0d-94d7-ef420bc9a155-kube-api-access-6dxk8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:22:58.101727 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.101553    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:22:58.101727 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.101576    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:22:58.101981 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.101960    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:22:58.102046 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.101996    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:22:58.111389 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.111365    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dxk8\" (UniqueName: \"kubernetes.io/projected/330d0380-4c84-4e0d-94d7-ef420bc9a155-kube-api-access-6dxk8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:22:58.192146 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.192110    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:22:58.326592 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.326567    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"]
Jul 17 09:22:58.863318 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:22:58.863279    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr" event={"ID":"330d0380-4c84-4e0d-94d7-ef420bc9a155","Type":"ContainerStarted","Data":"5ce5b2b18bd29f36a9b6c6da7cb3ea3fa0c128f859d37b96b14f03a9c22af9d0"}
Jul 17 09:23:05.892890 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:05.892805    2571 generic.go:356] "Generic (PLEG): container finished" podID="330d0380-4c84-4e0d-94d7-ef420bc9a155" containerID="a2db7969a154aabbba75ee8df1fb43c11c4eeb22d14f50904f8285c37de6881a" exitCode=0
Jul 17 09:23:05.893251 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:05.892897    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr" event={"ID":"330d0380-4c84-4e0d-94d7-ef420bc9a155","Type":"ContainerDied","Data":"a2db7969a154aabbba75ee8df1fb43c11c4eeb22d14f50904f8285c37de6881a"}
Jul 17 09:23:14.928673 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:14.928633    2571 generic.go:356] "Generic (PLEG): container finished" podID="330d0380-4c84-4e0d-94d7-ef420bc9a155" containerID="a345ea145d19810c3c667dd45a0e887ea20e3ca9581995c4e856b78b5de8ed3b" exitCode=0
Jul 17 09:23:14.929128 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:14.928717    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr" event={"ID":"330d0380-4c84-4e0d-94d7-ef420bc9a155","Type":"ContainerDied","Data":"a345ea145d19810c3c667dd45a0e887ea20e3ca9581995c4e856b78b5de8ed3b"}
Jul 17 09:23:22.963350 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:22.963313    2571 generic.go:356] "Generic (PLEG): container finished" podID="330d0380-4c84-4e0d-94d7-ef420bc9a155" containerID="bb2049265f5b0f1260dbf981782497f3a62342737c3ef4940557c46ae6f4ec47" exitCode=0
Jul 17 09:23:22.963724 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:22.963391    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr" event={"ID":"330d0380-4c84-4e0d-94d7-ef420bc9a155","Type":"ContainerDied","Data":"bb2049265f5b0f1260dbf981782497f3a62342737c3ef4940557c46ae6f4ec47"}
Jul 17 09:23:24.092998 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.092974    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:23:24.219995 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.219918    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-util\") pod \"330d0380-4c84-4e0d-94d7-ef420bc9a155\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") "
Jul 17 09:23:24.220126 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.220008    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-bundle\") pod \"330d0380-4c84-4e0d-94d7-ef420bc9a155\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") "
Jul 17 09:23:24.220126 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.220032    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dxk8\" (UniqueName: \"kubernetes.io/projected/330d0380-4c84-4e0d-94d7-ef420bc9a155-kube-api-access-6dxk8\") pod \"330d0380-4c84-4e0d-94d7-ef420bc9a155\" (UID: \"330d0380-4c84-4e0d-94d7-ef420bc9a155\") "
Jul 17 09:23:24.220817 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.220786    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-bundle" (OuterVolumeSpecName: "bundle") pod "330d0380-4c84-4e0d-94d7-ef420bc9a155" (UID: "330d0380-4c84-4e0d-94d7-ef420bc9a155"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:23:24.222308 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.222280    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/330d0380-4c84-4e0d-94d7-ef420bc9a155-kube-api-access-6dxk8" (OuterVolumeSpecName: "kube-api-access-6dxk8") pod "330d0380-4c84-4e0d-94d7-ef420bc9a155" (UID: "330d0380-4c84-4e0d-94d7-ef420bc9a155"). InnerVolumeSpecName "kube-api-access-6dxk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:23:24.224787 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.224767    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-util" (OuterVolumeSpecName: "util") pod "330d0380-4c84-4e0d-94d7-ef420bc9a155" (UID: "330d0380-4c84-4e0d-94d7-ef420bc9a155"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:23:24.320505 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.320447    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-bundle\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:23:24.320505 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.320499    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dxk8\" (UniqueName: \"kubernetes.io/projected/330d0380-4c84-4e0d-94d7-ef420bc9a155-kube-api-access-6dxk8\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:23:24.320505 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.320510    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/330d0380-4c84-4e0d-94d7-ef420bc9a155-util\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:23:24.973333 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.973306    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr"
Jul 17 09:23:24.973333 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.973328    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409qbxr" event={"ID":"330d0380-4c84-4e0d-94d7-ef420bc9a155","Type":"ContainerDied","Data":"5ce5b2b18bd29f36a9b6c6da7cb3ea3fa0c128f859d37b96b14f03a9c22af9d0"}
Jul 17 09:23:24.973548 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:24.973366    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ce5b2b18bd29f36a9b6c6da7cb3ea3fa0c128f859d37b96b14f03a9c22af9d0"
Jul 17 09:23:31.591443 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.591410    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2"]
Jul 17 09:23:31.591876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.591718    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="330d0380-4c84-4e0d-94d7-ef420bc9a155" containerName="util"
Jul 17 09:23:31.591876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.591730    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="330d0380-4c84-4e0d-94d7-ef420bc9a155" containerName="util"
Jul 17 09:23:31.591876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.591746    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="330d0380-4c84-4e0d-94d7-ef420bc9a155" containerName="extract"
Jul 17 09:23:31.591876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.591751    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="330d0380-4c84-4e0d-94d7-ef420bc9a155" containerName="extract"
Jul 17 09:23:31.591876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.591765    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="330d0380-4c84-4e0d-94d7-ef420bc9a155" containerName="pull"
Jul 17 09:23:31.591876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.591770    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="330d0380-4c84-4e0d-94d7-ef420bc9a155" containerName="pull"
Jul 17 09:23:31.591876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.591813    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="330d0380-4c84-4e0d-94d7-ef420bc9a155" containerName="extract"
Jul 17 09:23:31.594564 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.594548    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2"
Jul 17 09:23:31.604458 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.604436    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 17 09:23:31.604834 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.604817    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-fv9ss\""
Jul 17 09:23:31.615432 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.615413    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:23:31.627938 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.627908    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2"]
Jul 17 09:23:31.680751 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.680716    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw2dn\" (UniqueName: \"kubernetes.io/projected/9df857cc-a5af-4e13-8b51-146fc1df52bd-kube-api-access-mw2dn\") pod \"cert-manager-operator-controller-manager-55c444bf86-z7xb2\" (UID: \"9df857cc-a5af-4e13-8b51-146fc1df52bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2"
Jul 17 09:23:31.680931 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.680774    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9df857cc-a5af-4e13-8b51-146fc1df52bd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z7xb2\" (UID: \"9df857cc-a5af-4e13-8b51-146fc1df52bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2"
Jul 17 09:23:31.781205 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.781168    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9df857cc-a5af-4e13-8b51-146fc1df52bd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z7xb2\" (UID: \"9df857cc-a5af-4e13-8b51-146fc1df52bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2"
Jul 17 09:23:31.781367 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.781249    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw2dn\" (UniqueName: \"kubernetes.io/projected/9df857cc-a5af-4e13-8b51-146fc1df52bd-kube-api-access-mw2dn\") pod \"cert-manager-operator-controller-manager-55c444bf86-z7xb2\" (UID: \"9df857cc-a5af-4e13-8b51-146fc1df52bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2"
Jul 17 09:23:31.781658 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.781639    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9df857cc-a5af-4e13-8b51-146fc1df52bd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-z7xb2\" (UID: \"9df857cc-a5af-4e13-8b51-146fc1df52bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2"
Jul 17 09:23:31.790515 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.790488    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw2dn\" (UniqueName: \"kubernetes.io/projected/9df857cc-a5af-4e13-8b51-146fc1df52bd-kube-api-access-mw2dn\") pod \"cert-manager-operator-controller-manager-55c444bf86-z7xb2\" (UID: \"9df857cc-a5af-4e13-8b51-146fc1df52bd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2"
Jul 17 09:23:31.903644 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:31.903552    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2"
Jul 17 09:23:32.043766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:32.043690    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2"]
Jul 17 09:23:32.048961 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:23:32.048868    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df857cc_a5af_4e13_8b51_146fc1df52bd.slice/crio-7abdfcb320740238d3db6d7578db2bed381600a4b5310adcf5904effb2230615 WatchSource:0}: Error finding container 7abdfcb320740238d3db6d7578db2bed381600a4b5310adcf5904effb2230615: Status 404 returned error can't find the container with id 7abdfcb320740238d3db6d7578db2bed381600a4b5310adcf5904effb2230615
Jul 17 09:23:33.008764 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:33.008728    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2" event={"ID":"9df857cc-a5af-4e13-8b51-146fc1df52bd","Type":"ContainerStarted","Data":"7abdfcb320740238d3db6d7578db2bed381600a4b5310adcf5904effb2230615"}
Jul 17 09:23:37.266281 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:37.266244    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:23:37.266281 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:37.266265    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:23:37.271840 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:37.271810    2571 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 09:23:50.082488 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.082365    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2" event={"ID":"9df857cc-a5af-4e13-8b51-146fc1df52bd","Type":"ContainerStarted","Data":"224e313a4ae6b726a5857579a801976887524d53eec70b3448ab8a2567a4131f"}
Jul 17 09:23:50.106144 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.106087    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-z7xb2" podStartSLOduration=1.421621077 podStartE2EDuration="19.106068494s" totalImagesPullingTime="17.684447417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:23:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:23:50.10321174 +0000 UTC m=+314.168032463" watchObservedRunningTime="2026-07-17 09:23:50.106068494 +0000 UTC m=+314.170889215"
Jul 17 09:23:50.729401 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.729366    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"]
Jul 17 09:23:50.733101 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.733076    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:23:50.736304 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.736284    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-97w4q\""
Jul 17 09:23:50.736629 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.736614    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 17 09:23:50.737309 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.737248    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 17 09:23:50.745976 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.745951    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"]
Jul 17 09:23:50.840900 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.840862    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:23:50.840900 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.840896    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:23:50.841100 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.840917    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pff9z\" (UniqueName: \"kubernetes.io/projected/1c23bd24-89f5-4d91-aabe-2c7459a84270-kube-api-access-pff9z\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:23:50.942439 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.942399    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:23:50.942439 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.942437    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:23:50.942710 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.942455    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pff9z\" (UniqueName: \"kubernetes.io/projected/1c23bd24-89f5-4d91-aabe-2c7459a84270-kube-api-access-pff9z\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:23:50.942887 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.942865    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:23:50.942957 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.942917    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:23:50.961577 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:50.961543    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pff9z\" (UniqueName: \"kubernetes.io/projected/1c23bd24-89f5-4d91-aabe-2c7459a84270-kube-api-access-pff9z\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:23:51.043155 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:51.043058    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:23:51.175911 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:51.175874    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"]
Jul 17 09:23:51.178755 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:23:51.178729    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c23bd24_89f5_4d91_aabe_2c7459a84270.slice/crio-550f2df9294a88b744fce628a934037f901802ba522651ebbe496b8fbae0f7a0 WatchSource:0}: Error finding container 550f2df9294a88b744fce628a934037f901802ba522651ebbe496b8fbae0f7a0: Status 404 returned error can't find the container with id 550f2df9294a88b744fce628a934037f901802ba522651ebbe496b8fbae0f7a0
Jul 17 09:23:52.092477 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:52.092424    2571 generic.go:356] "Generic (PLEG): container finished" podID="1c23bd24-89f5-4d91-aabe-2c7459a84270" containerID="0bd9e4581642ee587c8bc9665849f98ce9c5cd57b1bbcbd81fb64d9fd2f397d3" exitCode=0
Jul 17 09:23:52.092803 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:52.092511    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr" event={"ID":"1c23bd24-89f5-4d91-aabe-2c7459a84270","Type":"ContainerDied","Data":"0bd9e4581642ee587c8bc9665849f98ce9c5cd57b1bbcbd81fb64d9fd2f397d3"}
Jul 17 09:23:52.092803 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:52.092569    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr" event={"ID":"1c23bd24-89f5-4d91-aabe-2c7459a84270","Type":"ContainerStarted","Data":"550f2df9294a88b744fce628a934037f901802ba522651ebbe496b8fbae0f7a0"}
Jul 17 09:23:52.093605 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:52.093587    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:23:57.301990 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.301950    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bsdt9"]
Jul 17 09:23:57.304298 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.304276    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9"
Jul 17 09:23:57.307009 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.306988    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 17 09:23:57.308000 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.307981    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-vpb9c\""
Jul 17 09:23:57.308112 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.308015    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 17 09:23:57.319195 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.319167    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bsdt9"]
Jul 17 09:23:57.497795 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.497755    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41c57185-4bc5-4ab6-99fe-f2bdf1362c9d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bsdt9\" (UID: \"41c57185-4bc5-4ab6-99fe-f2bdf1362c9d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9"
Jul 17 09:23:57.497795 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.497795    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq5w8\" (UniqueName: \"kubernetes.io/projected/41c57185-4bc5-4ab6-99fe-f2bdf1362c9d-kube-api-access-mq5w8\") pod \"cert-manager-webhook-6486ffd997-bsdt9\" (UID: \"41c57185-4bc5-4ab6-99fe-f2bdf1362c9d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9"
Jul 17 09:23:57.599321 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.599229    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq5w8\" (UniqueName: \"kubernetes.io/projected/41c57185-4bc5-4ab6-99fe-f2bdf1362c9d-kube-api-access-mq5w8\") pod \"cert-manager-webhook-6486ffd997-bsdt9\" (UID: \"41c57185-4bc5-4ab6-99fe-f2bdf1362c9d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9"
Jul 17 09:23:57.599496 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.599362    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41c57185-4bc5-4ab6-99fe-f2bdf1362c9d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bsdt9\" (UID: \"41c57185-4bc5-4ab6-99fe-f2bdf1362c9d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9"
Jul 17 09:23:57.607946 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.607922    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/41c57185-4bc5-4ab6-99fe-f2bdf1362c9d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bsdt9\" (UID: \"41c57185-4bc5-4ab6-99fe-f2bdf1362c9d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9"
Jul 17 09:23:57.608667 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.608649    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq5w8\" (UniqueName: \"kubernetes.io/projected/41c57185-4bc5-4ab6-99fe-f2bdf1362c9d-kube-api-access-mq5w8\") pod \"cert-manager-webhook-6486ffd997-bsdt9\" (UID: \"41c57185-4bc5-4ab6-99fe-f2bdf1362c9d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9"
Jul 17 09:23:57.632511 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.632480    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9"
Jul 17 09:23:57.762073 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:57.762048    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bsdt9"]
Jul 17 09:23:57.764140 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:23:57.764115    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41c57185_4bc5_4ab6_99fe_f2bdf1362c9d.slice/crio-897d1b7ede7200b2af1d79de8dd796a5cbb164b4b68d68714d112c1c6f39ec20 WatchSource:0}: Error finding container 897d1b7ede7200b2af1d79de8dd796a5cbb164b4b68d68714d112c1c6f39ec20: Status 404 returned error can't find the container with id 897d1b7ede7200b2af1d79de8dd796a5cbb164b4b68d68714d112c1c6f39ec20
Jul 17 09:23:58.118612 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:23:58.118577    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9" event={"ID":"41c57185-4bc5-4ab6-99fe-f2bdf1362c9d","Type":"ContainerStarted","Data":"897d1b7ede7200b2af1d79de8dd796a5cbb164b4b68d68714d112c1c6f39ec20"}
Jul 17 09:24:01.133501 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:01.133437    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9" event={"ID":"41c57185-4bc5-4ab6-99fe-f2bdf1362c9d","Type":"ContainerStarted","Data":"f652986aea32afa6607278f0cc488636e3c23ccdb5ed757c4833906c4aa321e8"}
Jul 17 09:24:01.133911 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:01.133590    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9"
Jul 17 09:24:01.154141 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:01.154071    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9" podStartSLOduration=1.748134167 podStartE2EDuration="4.154050464s" totalImagesPullingTime="2.405916297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:23:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:24:01.152655606 +0000 UTC m=+325.217476329" watchObservedRunningTime="2026-07-17 09:24:01.154050464 +0000 UTC m=+325.218871188"
Jul 17 09:24:02.139586 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.139551    2571 generic.go:356] "Generic (PLEG): container finished" podID="1c23bd24-89f5-4d91-aabe-2c7459a84270" containerID="77c3522905bd458ae6db778e58a5d8caa3c2844522270200ad1c4c79ce26df83" exitCode=0
Jul 17 09:24:02.140071 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.139631    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr" event={"ID":"1c23bd24-89f5-4d91-aabe-2c7459a84270","Type":"ContainerDied","Data":"77c3522905bd458ae6db778e58a5d8caa3c2844522270200ad1c4c79ce26df83"}
Jul 17 09:24:02.183713 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.183691    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z646p"]
Jul 17 09:24:02.186378 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.186359    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z646p"
Jul 17 09:24:02.189060 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.189042    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-74bsq\""
Jul 17 09:24:02.201565 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.201539    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z646p"]
Jul 17 09:24:02.236455 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.236424    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8zsd\" (UniqueName: \"kubernetes.io/projected/dd9ac242-75bb-4fcc-a337-25b286d62306-kube-api-access-r8zsd\") pod \"cert-manager-55b4d954d5-z646p\" (UID: \"dd9ac242-75bb-4fcc-a337-25b286d62306\") " pod="cert-manager/cert-manager-55b4d954d5-z646p"
Jul 17 09:24:02.236652 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.236511    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd9ac242-75bb-4fcc-a337-25b286d62306-bound-sa-token\") pod \"cert-manager-55b4d954d5-z646p\" (UID: \"dd9ac242-75bb-4fcc-a337-25b286d62306\") " pod="cert-manager/cert-manager-55b4d954d5-z646p"
Jul 17 09:24:02.337560 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.337513    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8zsd\" (UniqueName: \"kubernetes.io/projected/dd9ac242-75bb-4fcc-a337-25b286d62306-kube-api-access-r8zsd\") pod \"cert-manager-55b4d954d5-z646p\" (UID: \"dd9ac242-75bb-4fcc-a337-25b286d62306\") " pod="cert-manager/cert-manager-55b4d954d5-z646p"
Jul 17 09:24:02.337749 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.337590    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd9ac242-75bb-4fcc-a337-25b286d62306-bound-sa-token\") pod \"cert-manager-55b4d954d5-z646p\" (UID: \"dd9ac242-75bb-4fcc-a337-25b286d62306\") " pod="cert-manager/cert-manager-55b4d954d5-z646p"
Jul 17 09:24:02.346424 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.346393    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8zsd\" (UniqueName: \"kubernetes.io/projected/dd9ac242-75bb-4fcc-a337-25b286d62306-kube-api-access-r8zsd\") pod \"cert-manager-55b4d954d5-z646p\" (UID: \"dd9ac242-75bb-4fcc-a337-25b286d62306\") " pod="cert-manager/cert-manager-55b4d954d5-z646p"
Jul 17 09:24:02.346424 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.346403    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dd9ac242-75bb-4fcc-a337-25b286d62306-bound-sa-token\") pod \"cert-manager-55b4d954d5-z646p\" (UID: \"dd9ac242-75bb-4fcc-a337-25b286d62306\") " pod="cert-manager/cert-manager-55b4d954d5-z646p"
Jul 17 09:24:02.495341 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.495305    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z646p"
Jul 17 09:24:02.631561 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:02.631438    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z646p"]
Jul 17 09:24:02.635192 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:24:02.635159    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd9ac242_75bb_4fcc_a337_25b286d62306.slice/crio-66fc034203a0f58785e97846a652c00b347fb7d7e26dbd19b61b2c891b3e243c WatchSource:0}: Error finding container 66fc034203a0f58785e97846a652c00b347fb7d7e26dbd19b61b2c891b3e243c: Status 404 returned error can't find the container with id 66fc034203a0f58785e97846a652c00b347fb7d7e26dbd19b61b2c891b3e243c
Jul 17 09:24:03.145982 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:03.145941    2571 generic.go:356] "Generic (PLEG): container finished" podID="1c23bd24-89f5-4d91-aabe-2c7459a84270" containerID="a210afec8c34cc6613fb0c967bbdf88ec41a533e15890a658e6c89734e7cf097" exitCode=0
Jul 17 09:24:03.146424 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:03.146023    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr" event={"ID":"1c23bd24-89f5-4d91-aabe-2c7459a84270","Type":"ContainerDied","Data":"a210afec8c34cc6613fb0c967bbdf88ec41a533e15890a658e6c89734e7cf097"}
Jul 17 09:24:03.147395 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:03.147365    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z646p" event={"ID":"dd9ac242-75bb-4fcc-a337-25b286d62306","Type":"ContainerStarted","Data":"ce853a8c0a31e468671accbdb15a6c3063cbb44bb5fe9a8348200d5510265e50"}
Jul 17 09:24:03.147395 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:03.147396    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z646p" event={"ID":"dd9ac242-75bb-4fcc-a337-25b286d62306","Type":"ContainerStarted","Data":"66fc034203a0f58785e97846a652c00b347fb7d7e26dbd19b61b2c891b3e243c"}
Jul 17 09:24:03.187988 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:03.187935    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-z646p" podStartSLOduration=1.18792209 podStartE2EDuration="1.18792209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:24:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:24:03.187434587 +0000 UTC m=+327.252255308" watchObservedRunningTime="2026-07-17 09:24:03.18792209 +0000 UTC m=+327.252742811"
Jul 17 09:24:04.281589 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:04.281561    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:24:04.353458 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:04.353425    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-util\") pod \"1c23bd24-89f5-4d91-aabe-2c7459a84270\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") "
Jul 17 09:24:04.353458 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:04.353483    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-bundle\") pod \"1c23bd24-89f5-4d91-aabe-2c7459a84270\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") "
Jul 17 09:24:04.353724 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:04.353548    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pff9z\" (UniqueName: \"kubernetes.io/projected/1c23bd24-89f5-4d91-aabe-2c7459a84270-kube-api-access-pff9z\") pod \"1c23bd24-89f5-4d91-aabe-2c7459a84270\" (UID: \"1c23bd24-89f5-4d91-aabe-2c7459a84270\") "
Jul 17 09:24:04.353977 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:04.353945    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-bundle" (OuterVolumeSpecName: "bundle") pod "1c23bd24-89f5-4d91-aabe-2c7459a84270" (UID: "1c23bd24-89f5-4d91-aabe-2c7459a84270"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:24:04.355722 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:04.355699    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c23bd24-89f5-4d91-aabe-2c7459a84270-kube-api-access-pff9z" (OuterVolumeSpecName: "kube-api-access-pff9z") pod "1c23bd24-89f5-4d91-aabe-2c7459a84270" (UID: "1c23bd24-89f5-4d91-aabe-2c7459a84270"). InnerVolumeSpecName "kube-api-access-pff9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:24:04.357980 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:04.357942    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-util" (OuterVolumeSpecName: "util") pod "1c23bd24-89f5-4d91-aabe-2c7459a84270" (UID: "1c23bd24-89f5-4d91-aabe-2c7459a84270"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:24:04.454663 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:04.454633    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-util\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:24:04.454663 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:04.454660    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c23bd24-89f5-4d91-aabe-2c7459a84270-bundle\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:24:04.454847 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:04.454674    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pff9z\" (UniqueName: \"kubernetes.io/projected/1c23bd24-89f5-4d91-aabe-2c7459a84270-kube-api-access-pff9z\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:24:05.159063 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:05.159031    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr"
Jul 17 09:24:05.159241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:05.159025    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq7plr" event={"ID":"1c23bd24-89f5-4d91-aabe-2c7459a84270","Type":"ContainerDied","Data":"550f2df9294a88b744fce628a934037f901802ba522651ebbe496b8fbae0f7a0"}
Jul 17 09:24:05.159241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:05.159159    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="550f2df9294a88b744fce628a934037f901802ba522651ebbe496b8fbae0f7a0"
Jul 17 09:24:07.141690 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:07.141659    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bsdt9"
Jul 17 09:24:12.956663 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.956626    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"]
Jul 17 09:24:12.957038 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.956990    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c23bd24-89f5-4d91-aabe-2c7459a84270" containerName="pull"
Jul 17 09:24:12.957038 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.957003    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c23bd24-89f5-4d91-aabe-2c7459a84270" containerName="pull"
Jul 17 09:24:12.957038 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.957014    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c23bd24-89f5-4d91-aabe-2c7459a84270" containerName="extract"
Jul 17 09:24:12.957038 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.957019    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c23bd24-89f5-4d91-aabe-2c7459a84270" containerName="extract"
Jul 17 09:24:12.957038 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.957026    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c23bd24-89f5-4d91-aabe-2c7459a84270" containerName="util"
Jul 17 09:24:12.957038 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.957032    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c23bd24-89f5-4d91-aabe-2c7459a84270" containerName="util"
Jul 17 09:24:12.957251 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.957080    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c23bd24-89f5-4d91-aabe-2c7459a84270" containerName="extract"
Jul 17 09:24:12.960299 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.960283    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:12.963073 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.963050    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 17 09:24:12.964131 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.964112    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 17 09:24:12.964249 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.964141    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-97w4q\""
Jul 17 09:24:12.969270 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:12.969245    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"]
Jul 17 09:24:13.023657 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:13.023620    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bxl2\" (UniqueName: \"kubernetes.io/projected/8747d594-3e07-47ef-be59-6380edc250c6-kube-api-access-2bxl2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:13.023814 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:13.023665    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:13.023814 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:13.023765    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:13.124882 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:13.124848    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:13.125033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:13.124904    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bxl2\" (UniqueName: \"kubernetes.io/projected/8747d594-3e07-47ef-be59-6380edc250c6-kube-api-access-2bxl2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:13.125033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:13.124930    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:13.125247 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:13.125226    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:13.125306 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:13.125286    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:13.134076 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:13.134046    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bxl2\" (UniqueName: \"kubernetes.io/projected/8747d594-3e07-47ef-be59-6380edc250c6-kube-api-access-2bxl2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:13.270097 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:13.270001    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:13.402572 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:13.402546    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"]
Jul 17 09:24:13.404941 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:24:13.404911    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8747d594_3e07_47ef_be59_6380edc250c6.slice/crio-18ec2f0b6ac0fce7303da98586d70e0531ae4f4efbe6d5b1415cf7e0dfa944ba WatchSource:0}: Error finding container 18ec2f0b6ac0fce7303da98586d70e0531ae4f4efbe6d5b1415cf7e0dfa944ba: Status 404 returned error can't find the container with id 18ec2f0b6ac0fce7303da98586d70e0531ae4f4efbe6d5b1415cf7e0dfa944ba
Jul 17 09:24:14.198760 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:14.198725    2571 generic.go:356] "Generic (PLEG): container finished" podID="8747d594-3e07-47ef-be59-6380edc250c6" containerID="de04018898ec6b8efb336b95cfe3140f84da965625011366eade80275e1c7352" exitCode=0
Jul 17 09:24:14.199210 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:14.198763    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8" event={"ID":"8747d594-3e07-47ef-be59-6380edc250c6","Type":"ContainerDied","Data":"de04018898ec6b8efb336b95cfe3140f84da965625011366eade80275e1c7352"}
Jul 17 09:24:14.199210 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:14.198842    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8" event={"ID":"8747d594-3e07-47ef-be59-6380edc250c6","Type":"ContainerStarted","Data":"18ec2f0b6ac0fce7303da98586d70e0531ae4f4efbe6d5b1415cf7e0dfa944ba"}
Jul 17 09:24:15.204678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:15.204637    2571 generic.go:356] "Generic (PLEG): container finished" podID="8747d594-3e07-47ef-be59-6380edc250c6" containerID="a05eaba644a557ea04b78f3639edb8bfd13b8221f39775972eb560c0d9793d4a" exitCode=0
Jul 17 09:24:15.205000 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:15.204719    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8" event={"ID":"8747d594-3e07-47ef-be59-6380edc250c6","Type":"ContainerDied","Data":"a05eaba644a557ea04b78f3639edb8bfd13b8221f39775972eb560c0d9793d4a"}
Jul 17 09:24:16.211519 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:16.211489    2571 generic.go:356] "Generic (PLEG): container finished" podID="8747d594-3e07-47ef-be59-6380edc250c6" containerID="2fb51758c7622965494bcd065d16ae99764a80fe45b4358f0d63d9671b40c03c" exitCode=0
Jul 17 09:24:16.211869 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:16.211573    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8" event={"ID":"8747d594-3e07-47ef-be59-6380edc250c6","Type":"ContainerDied","Data":"2fb51758c7622965494bcd065d16ae99764a80fe45b4358f0d63d9671b40c03c"}
Jul 17 09:24:17.338234 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:17.338212    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:17.460020 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:17.459983    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bxl2\" (UniqueName: \"kubernetes.io/projected/8747d594-3e07-47ef-be59-6380edc250c6-kube-api-access-2bxl2\") pod \"8747d594-3e07-47ef-be59-6380edc250c6\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") "
Jul 17 09:24:17.460020 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:17.460022    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-util\") pod \"8747d594-3e07-47ef-be59-6380edc250c6\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") "
Jul 17 09:24:17.460271 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:17.460103    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-bundle\") pod \"8747d594-3e07-47ef-be59-6380edc250c6\" (UID: \"8747d594-3e07-47ef-be59-6380edc250c6\") "
Jul 17 09:24:17.460839 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:17.460810    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-bundle" (OuterVolumeSpecName: "bundle") pod "8747d594-3e07-47ef-be59-6380edc250c6" (UID: "8747d594-3e07-47ef-be59-6380edc250c6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:24:17.462105 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:17.462073    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8747d594-3e07-47ef-be59-6380edc250c6-kube-api-access-2bxl2" (OuterVolumeSpecName: "kube-api-access-2bxl2") pod "8747d594-3e07-47ef-be59-6380edc250c6" (UID: "8747d594-3e07-47ef-be59-6380edc250c6"). InnerVolumeSpecName "kube-api-access-2bxl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:24:17.465865 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:17.465834    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-util" (OuterVolumeSpecName: "util") pod "8747d594-3e07-47ef-be59-6380edc250c6" (UID: "8747d594-3e07-47ef-be59-6380edc250c6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:24:17.560678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:17.560592    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bxl2\" (UniqueName: \"kubernetes.io/projected/8747d594-3e07-47ef-be59-6380edc250c6-kube-api-access-2bxl2\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:24:17.560678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:17.560626    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-util\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:24:17.560678 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:17.560635    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8747d594-3e07-47ef-be59-6380edc250c6-bundle\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:24:18.222744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:18.222705    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8" event={"ID":"8747d594-3e07-47ef-be59-6380edc250c6","Type":"ContainerDied","Data":"18ec2f0b6ac0fce7303da98586d70e0531ae4f4efbe6d5b1415cf7e0dfa944ba"}
Jul 17 09:24:18.222744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:18.222744    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9kzm8"
Jul 17 09:24:18.223014 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:18.222760    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="18ec2f0b6ac0fce7303da98586d70e0531ae4f4efbe6d5b1415cf7e0dfa944ba"
Jul 17 09:24:29.840806 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.840728    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"]
Jul 17 09:24:29.841153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.841033    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8747d594-3e07-47ef-be59-6380edc250c6" containerName="pull"
Jul 17 09:24:29.841153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.841043    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8747d594-3e07-47ef-be59-6380edc250c6" containerName="pull"
Jul 17 09:24:29.841153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.841060    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8747d594-3e07-47ef-be59-6380edc250c6" containerName="extract"
Jul 17 09:24:29.841153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.841066    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8747d594-3e07-47ef-be59-6380edc250c6" containerName="extract"
Jul 17 09:24:29.841153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.841074    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8747d594-3e07-47ef-be59-6380edc250c6" containerName="util"
Jul 17 09:24:29.841153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.841079    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8747d594-3e07-47ef-be59-6380edc250c6" containerName="util"
Jul 17 09:24:29.841153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.841136    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8747d594-3e07-47ef-be59-6380edc250c6" containerName="extract"
Jul 17 09:24:29.844254 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.844232    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:29.846996 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.846971    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 09:24:29.847229 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.847210    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pljhm\""
Jul 17 09:24:29.847320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.847297    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 09:24:29.847386 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.847296    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 09:24:29.847428 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.847388    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 09:24:29.874536 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.874506    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"]
Jul 17 09:24:29.968958 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.968914    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b715bf43-21ac-4495-9a18-fb596c629ab0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:29.968958 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.968957    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b715bf43-21ac-4495-9a18-fb596c629ab0-manifests\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:29.969225 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.968977    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b715bf43-21ac-4495-9a18-fb596c629ab0-tmp\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:29.969225 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.969125    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b715bf43-21ac-4495-9a18-fb596c629ab0-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:29.969225 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:29.969170    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84w64\" (UniqueName: \"kubernetes.io/projected/b715bf43-21ac-4495-9a18-fb596c629ab0-kube-api-access-84w64\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.069900 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.069862    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b715bf43-21ac-4495-9a18-fb596c629ab0-manifests\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.069900 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.069901    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b715bf43-21ac-4495-9a18-fb596c629ab0-tmp\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.070157 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.069947    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b715bf43-21ac-4495-9a18-fb596c629ab0-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.070157 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.069966    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84w64\" (UniqueName: \"kubernetes.io/projected/b715bf43-21ac-4495-9a18-fb596c629ab0-kube-api-access-84w64\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.070157 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.070012    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b715bf43-21ac-4495-9a18-fb596c629ab0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.070357 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.070334    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b715bf43-21ac-4495-9a18-fb596c629ab0-manifests\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.072323 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.072293    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b715bf43-21ac-4495-9a18-fb596c629ab0-tmp\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.072575 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.072556    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b715bf43-21ac-4495-9a18-fb596c629ab0-webhook-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.072625 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.072592    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b715bf43-21ac-4495-9a18-fb596c629ab0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.078912 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.078881    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84w64\" (UniqueName: \"kubernetes.io/projected/b715bf43-21ac-4495-9a18-fb596c629ab0-kube-api-access-84w64\") pod \"opendatahub-operator-controller-manager-76fdbb8988-fhn4b\" (UID: \"b715bf43-21ac-4495-9a18-fb596c629ab0\") " pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.154907 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.154820    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:30.292119 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:30.292094    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"]
Jul 17 09:24:30.294700 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:24:30.294673    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb715bf43_21ac_4495_9a18_fb596c629ab0.slice/crio-55a93d9f7bbcf2cbc23ab87f16b995e9adda66dfad656ba3ad4100ac5e879c91 WatchSource:0}: Error finding container 55a93d9f7bbcf2cbc23ab87f16b995e9adda66dfad656ba3ad4100ac5e879c91: Status 404 returned error can't find the container with id 55a93d9f7bbcf2cbc23ab87f16b995e9adda66dfad656ba3ad4100ac5e879c91
Jul 17 09:24:31.284200 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:31.284161    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b" event={"ID":"b715bf43-21ac-4495-9a18-fb596c629ab0","Type":"ContainerStarted","Data":"55a93d9f7bbcf2cbc23ab87f16b995e9adda66dfad656ba3ad4100ac5e879c91"}
Jul 17 09:24:34.301124 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:34.301087    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b" event={"ID":"b715bf43-21ac-4495-9a18-fb596c629ab0","Type":"ContainerStarted","Data":"817c554b76adcbbeee065ae32d99d8089601529375c22d357351958311c854e6"}
Jul 17 09:24:35.307858 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:35.307819    2571 generic.go:356] "Generic (PLEG): container finished" podID="b715bf43-21ac-4495-9a18-fb596c629ab0" containerID="817c554b76adcbbeee065ae32d99d8089601529375c22d357351958311c854e6" exitCode=0
Jul 17 09:24:35.308251 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:35.307861    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b" event={"ID":"b715bf43-21ac-4495-9a18-fb596c629ab0","Type":"ContainerDied","Data":"817c554b76adcbbeee065ae32d99d8089601529375c22d357351958311c854e6"}
Jul 17 09:24:36.314177 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:36.314137    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b" event={"ID":"b715bf43-21ac-4495-9a18-fb596c629ab0","Type":"ContainerStarted","Data":"fc7c38f645db152c4dc941d15369153caa7ec40ea1e8b7ebfc0519734690e1eb"}
Jul 17 09:24:36.314586 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:36.314247    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:36.340440 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:36.340372    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b" podStartSLOduration=2.967423621 podStartE2EDuration="7.34034964s" totalImagesPullingTime="3.306182117s" totalInitContainerRuntime="1.066743902s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:24:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:24:36.338445641 +0000 UTC m=+360.403266374" watchObservedRunningTime="2026-07-17 09:24:36.34034964 +0000 UTC m=+360.405170363"
Jul 17 09:24:47.321214 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:47.321181    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-76fdbb8988-fhn4b"
Jul 17 09:24:49.918001 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:49.917962    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jgsgk"]
Jul 17 09:24:49.924190 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:49.924155    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk"
Jul 17 09:24:49.927444 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:49.927420    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 17 09:24:49.927595 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:49.927421    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-29n5s\""
Jul 17 09:24:49.939575 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:49.939547    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jgsgk"]
Jul 17 09:24:50.034561 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.034522    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p5fm\" (UniqueName: \"kubernetes.io/projected/336327ee-2ed2-4751-8516-b8d54a64900f-kube-api-access-4p5fm\") pod \"odh-model-controller-7998c67dc4-jgsgk\" (UID: \"336327ee-2ed2-4751-8516-b8d54a64900f\") " pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk"
Jul 17 09:24:50.034741 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.034647    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/336327ee-2ed2-4751-8516-b8d54a64900f-cert\") pod \"odh-model-controller-7998c67dc4-jgsgk\" (UID: \"336327ee-2ed2-4751-8516-b8d54a64900f\") " pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk"
Jul 17 09:24:50.122837 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.122803    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"]
Jul 17 09:24:50.125116 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.125100    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:50.127934 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.127907    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 17 09:24:50.128054 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.127983    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-np6hb\""
Jul 17 09:24:50.128054 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.128006    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 09:24:50.128530 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.128508    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 17 09:24:50.128619 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.128551    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 09:24:50.135333 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.135313    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/336327ee-2ed2-4751-8516-b8d54a64900f-cert\") pod \"odh-model-controller-7998c67dc4-jgsgk\" (UID: \"336327ee-2ed2-4751-8516-b8d54a64900f\") " pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk"
Jul 17 09:24:50.135426 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.135365    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p5fm\" (UniqueName: \"kubernetes.io/projected/336327ee-2ed2-4751-8516-b8d54a64900f-kube-api-access-4p5fm\") pod \"odh-model-controller-7998c67dc4-jgsgk\" (UID: \"336327ee-2ed2-4751-8516-b8d54a64900f\") " pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk"
Jul 17 09:24:50.137654 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.137636    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/336327ee-2ed2-4751-8516-b8d54a64900f-cert\") pod \"odh-model-controller-7998c67dc4-jgsgk\" (UID: \"336327ee-2ed2-4751-8516-b8d54a64900f\") " pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk"
Jul 17 09:24:50.142230 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.142208    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"]
Jul 17 09:24:50.164529 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.164498    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p5fm\" (UniqueName: \"kubernetes.io/projected/336327ee-2ed2-4751-8516-b8d54a64900f-kube-api-access-4p5fm\") pod \"odh-model-controller-7998c67dc4-jgsgk\" (UID: \"336327ee-2ed2-4751-8516-b8d54a64900f\") " pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk"
Jul 17 09:24:50.234392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.234316    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk"
Jul 17 09:24:50.236273 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.236250    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83d184a8-32f9-4b5e-b244-06430e33b380-tmp\") pod \"kube-auth-proxy-7df6bb5dc-nn7bl\" (UID: \"83d184a8-32f9-4b5e-b244-06430e33b380\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:50.236378 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.236290    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83d184a8-32f9-4b5e-b244-06430e33b380-tls-certs\") pod \"kube-auth-proxy-7df6bb5dc-nn7bl\" (UID: \"83d184a8-32f9-4b5e-b244-06430e33b380\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:50.236378 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.236359    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fglv9\" (UniqueName: \"kubernetes.io/projected/83d184a8-32f9-4b5e-b244-06430e33b380-kube-api-access-fglv9\") pod \"kube-auth-proxy-7df6bb5dc-nn7bl\" (UID: \"83d184a8-32f9-4b5e-b244-06430e33b380\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:50.337041 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.337007    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fglv9\" (UniqueName: \"kubernetes.io/projected/83d184a8-32f9-4b5e-b244-06430e33b380-kube-api-access-fglv9\") pod \"kube-auth-proxy-7df6bb5dc-nn7bl\" (UID: \"83d184a8-32f9-4b5e-b244-06430e33b380\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:50.337222 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.337077    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83d184a8-32f9-4b5e-b244-06430e33b380-tmp\") pod \"kube-auth-proxy-7df6bb5dc-nn7bl\" (UID: \"83d184a8-32f9-4b5e-b244-06430e33b380\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:50.337222 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.337104    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83d184a8-32f9-4b5e-b244-06430e33b380-tls-certs\") pod \"kube-auth-proxy-7df6bb5dc-nn7bl\" (UID: \"83d184a8-32f9-4b5e-b244-06430e33b380\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:50.337222 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:24:50.337205    2571 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 17 09:24:50.337377 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:24:50.337267    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83d184a8-32f9-4b5e-b244-06430e33b380-tls-certs podName:83d184a8-32f9-4b5e-b244-06430e33b380 nodeName:}" failed. No retries permitted until 2026-07-17 09:24:50.837245387 +0000 UTC m=+374.902066087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/83d184a8-32f9-4b5e-b244-06430e33b380-tls-certs") pod "kube-auth-proxy-7df6bb5dc-nn7bl" (UID: "83d184a8-32f9-4b5e-b244-06430e33b380") : secret "kube-auth-proxy-tls" not found
Jul 17 09:24:50.339889 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.339844    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/83d184a8-32f9-4b5e-b244-06430e33b380-tmp\") pod \"kube-auth-proxy-7df6bb5dc-nn7bl\" (UID: \"83d184a8-32f9-4b5e-b244-06430e33b380\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:50.349557 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.349528    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fglv9\" (UniqueName: \"kubernetes.io/projected/83d184a8-32f9-4b5e-b244-06430e33b380-kube-api-access-fglv9\") pod \"kube-auth-proxy-7df6bb5dc-nn7bl\" (UID: \"83d184a8-32f9-4b5e-b244-06430e33b380\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:50.376444 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.376329    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jgsgk"]
Jul 17 09:24:50.378935 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:24:50.378907    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336327ee_2ed2_4751_8516_b8d54a64900f.slice/crio-0dbb7c7e13fe87b809f5a9d4e747f41453d7bb151d573ef892a62bd5cb32818e WatchSource:0}: Error finding container 0dbb7c7e13fe87b809f5a9d4e747f41453d7bb151d573ef892a62bd5cb32818e: Status 404 returned error can't find the container with id 0dbb7c7e13fe87b809f5a9d4e747f41453d7bb151d573ef892a62bd5cb32818e
Jul 17 09:24:50.744413 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.744376    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-g6z4w"]
Jul 17 09:24:50.747183 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.747157    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w"
Jul 17 09:24:50.751628 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.751600    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 17 09:24:50.751999 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.751981    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-7plhz\""
Jul 17 09:24:50.775431 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.775383    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-g6z4w"]
Jul 17 09:24:50.841380 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.841334    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxlsm\" (UniqueName: \"kubernetes.io/projected/aa670365-d71a-447d-98aa-74077e385b07-kube-api-access-kxlsm\") pod \"model-serving-api-54d89bd5fc-g6z4w\" (UID: \"aa670365-d71a-447d-98aa-74077e385b07\") " pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w"
Jul 17 09:24:50.841603 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.841402    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83d184a8-32f9-4b5e-b244-06430e33b380-tls-certs\") pod \"kube-auth-proxy-7df6bb5dc-nn7bl\" (UID: \"83d184a8-32f9-4b5e-b244-06430e33b380\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:50.841603 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.841491    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa670365-d71a-447d-98aa-74077e385b07-tls-certs\") pod \"model-serving-api-54d89bd5fc-g6z4w\" (UID: \"aa670365-d71a-447d-98aa-74077e385b07\") " pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w"
Jul 17 09:24:50.843826 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.843807    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/83d184a8-32f9-4b5e-b244-06430e33b380-tls-certs\") pod \"kube-auth-proxy-7df6bb5dc-nn7bl\" (UID: \"83d184a8-32f9-4b5e-b244-06430e33b380\") " pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:50.942906 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.942864    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxlsm\" (UniqueName: \"kubernetes.io/projected/aa670365-d71a-447d-98aa-74077e385b07-kube-api-access-kxlsm\") pod \"model-serving-api-54d89bd5fc-g6z4w\" (UID: \"aa670365-d71a-447d-98aa-74077e385b07\") " pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w"
Jul 17 09:24:50.943411 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.942977    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa670365-d71a-447d-98aa-74077e385b07-tls-certs\") pod \"model-serving-api-54d89bd5fc-g6z4w\" (UID: \"aa670365-d71a-447d-98aa-74077e385b07\") " pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w"
Jul 17 09:24:50.943411 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:24:50.943106    2571 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 17 09:24:50.943411 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:24:50.943166    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa670365-d71a-447d-98aa-74077e385b07-tls-certs podName:aa670365-d71a-447d-98aa-74077e385b07 nodeName:}" failed. No retries permitted until 2026-07-17 09:24:51.443145851 +0000 UTC m=+375.507966559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/aa670365-d71a-447d-98aa-74077e385b07-tls-certs") pod "model-serving-api-54d89bd5fc-g6z4w" (UID: "aa670365-d71a-447d-98aa-74077e385b07") : secret "model-serving-api-tls" not found
Jul 17 09:24:50.954913 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:50.954874    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxlsm\" (UniqueName: \"kubernetes.io/projected/aa670365-d71a-447d-98aa-74077e385b07-kube-api-access-kxlsm\") pod \"model-serving-api-54d89bd5fc-g6z4w\" (UID: \"aa670365-d71a-447d-98aa-74077e385b07\") " pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w"
Jul 17 09:24:51.035997 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:51.035901    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"
Jul 17 09:24:51.227337 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:51.227303    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl"]
Jul 17 09:24:51.230361 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:24:51.230326    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83d184a8_32f9_4b5e_b244_06430e33b380.slice/crio-f4dc495f023ce8b54c5f3b917dc7b86f0b0562ab7a96ffa62d8ad2e7f7a3cfcb WatchSource:0}: Error finding container f4dc495f023ce8b54c5f3b917dc7b86f0b0562ab7a96ffa62d8ad2e7f7a3cfcb: Status 404 returned error can't find the container with id f4dc495f023ce8b54c5f3b917dc7b86f0b0562ab7a96ffa62d8ad2e7f7a3cfcb
Jul 17 09:24:51.389886 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:51.389793    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl" event={"ID":"83d184a8-32f9-4b5e-b244-06430e33b380","Type":"ContainerStarted","Data":"f4dc495f023ce8b54c5f3b917dc7b86f0b0562ab7a96ffa62d8ad2e7f7a3cfcb"}
Jul 17 09:24:51.391486 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:51.391405    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk" event={"ID":"336327ee-2ed2-4751-8516-b8d54a64900f","Type":"ContainerStarted","Data":"0dbb7c7e13fe87b809f5a9d4e747f41453d7bb151d573ef892a62bd5cb32818e"}
Jul 17 09:24:51.448095 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:51.448054    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa670365-d71a-447d-98aa-74077e385b07-tls-certs\") pod \"model-serving-api-54d89bd5fc-g6z4w\" (UID: \"aa670365-d71a-447d-98aa-74077e385b07\") " pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w"
Jul 17 09:24:51.451265 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:51.451214    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa670365-d71a-447d-98aa-74077e385b07-tls-certs\") pod \"model-serving-api-54d89bd5fc-g6z4w\" (UID: \"aa670365-d71a-447d-98aa-74077e385b07\") " pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w"
Jul 17 09:24:51.658629 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:51.658549    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w"
Jul 17 09:24:51.821988 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:51.821908    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-g6z4w"]
Jul 17 09:24:53.180666 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:24:53.180626    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa670365_d71a_447d_98aa_74077e385b07.slice/crio-d7d6ead79ee31160182a58017feab7598dff2ca383aaff3322592a93bc69904c WatchSource:0}: Error finding container d7d6ead79ee31160182a58017feab7598dff2ca383aaff3322592a93bc69904c: Status 404 returned error can't find the container with id d7d6ead79ee31160182a58017feab7598dff2ca383aaff3322592a93bc69904c
Jul 17 09:24:53.403262 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:53.403212    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk" event={"ID":"336327ee-2ed2-4751-8516-b8d54a64900f","Type":"ContainerStarted","Data":"2d92231f4cb8c4c39c74cb51fbb2e66b9401e63fbbe50fdc472bc61cc349d579"}
Jul 17 09:24:53.403452 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:53.403325    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk"
Jul 17 09:24:53.404637 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:53.404605    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w" event={"ID":"aa670365-d71a-447d-98aa-74077e385b07","Type":"ContainerStarted","Data":"d7d6ead79ee31160182a58017feab7598dff2ca383aaff3322592a93bc69904c"}
Jul 17 09:24:53.428923 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:53.428867    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk" podStartSLOduration=1.592990277 podStartE2EDuration="4.428847858s" totalImagesPullingTime="2.835857581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:24:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:24:53.427589352 +0000 UTC m=+377.492410077" watchObservedRunningTime="2026-07-17 09:24:53.428847858 +0000 UTC m=+377.493668583"
Jul 17 09:24:54.411408 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:54.411310    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl" event={"ID":"83d184a8-32f9-4b5e-b244-06430e33b380","Type":"ContainerStarted","Data":"7e71b2627a6e8b4a657573ffa04eb3fdae375f9f9ffaafc8fd799b37abbeabad"}
Jul 17 09:24:54.432407 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:54.432345    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7df6bb5dc-nn7bl" podStartSLOduration=1.704262548 podStartE2EDuration="4.432330124s" totalImagesPullingTime="2.728067576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:24:54.430667772 +0000 UTC m=+378.495488495" watchObservedRunningTime="2026-07-17 09:24:54.432330124 +0000 UTC m=+378.497150845"
Jul 17 09:24:55.417216 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:55.417133    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w" event={"ID":"aa670365-d71a-447d-98aa-74077e385b07","Type":"ContainerStarted","Data":"eb7d60c50d72f4c2e7c850eb580da3ce9ba6a5cb94e261c34593f074aef69151"}
Jul 17 09:24:55.417602 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:55.417422    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w"
Jul 17 09:24:55.445540 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:24:55.445493    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w" podStartSLOduration=3.610228594 podStartE2EDuration="5.445453914s" totalImagesPullingTime="1.83522532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:24:55.444081747 +0000 UTC m=+379.508902468" watchObservedRunningTime="2026-07-17 09:24:55.445453914 +0000 UTC m=+379.510274636"
Jul 17 09:25:00.132233 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.132199    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"]
Jul 17 09:25:00.135116 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.135100    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.140356 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.140333    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 17 09:25:00.140986 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.140962    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kjwfh\""
Jul 17 09:25:00.141096 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.140991    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 09:25:00.141096 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.141002    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 17 09:25:00.141096 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.141000    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 17 09:25:00.161976 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.161949    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"]
Jul 17 09:25:00.223441 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.223407    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.223628 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.223456    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.223628 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.223532    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.223628 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.223566    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.223628 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.223609    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.223759 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.223639    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.223759 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.223657    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rszmx\" (UniqueName: \"kubernetes.io/projected/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-kube-api-access-rszmx\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.324291 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.324257    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.324291 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.324296    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.324569 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.324316    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rszmx\" (UniqueName: \"kubernetes.io/projected/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-kube-api-access-rszmx\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.324569 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.324349    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.324569 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.324381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.324569 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.324417    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.324569 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.324485    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.325346 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.325313    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.326862 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.326834    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.326971 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.326886    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.326971 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.326940    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.326971 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.326944    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.339079 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.339055    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rszmx\" (UniqueName: \"kubernetes.io/projected/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-kube-api-access-rszmx\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.339554 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.339535    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04aa59a4-2be5-498f-b3e6-6c1d9bf92551-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ktvxn\" (UID: \"04aa59a4-2be5-498f-b3e6-6c1d9bf92551\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.444471 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.444441    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:00.591141 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:00.591113    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"]
Jul 17 09:25:00.593448 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:25:00.593422    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04aa59a4_2be5_498f_b3e6_6c1d9bf92551.slice/crio-f5b2da72a400539629d22d9e261f5c0f5daf8d1f9c2177098c7678afa9b027f9 WatchSource:0}: Error finding container f5b2da72a400539629d22d9e261f5c0f5daf8d1f9c2177098c7678afa9b027f9: Status 404 returned error can't find the container with id f5b2da72a400539629d22d9e261f5c0f5daf8d1f9c2177098c7678afa9b027f9
Jul 17 09:25:01.446801 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:01.446730    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn" event={"ID":"04aa59a4-2be5-498f-b3e6-6c1d9bf92551","Type":"ContainerStarted","Data":"f5b2da72a400539629d22d9e261f5c0f5daf8d1f9c2177098c7678afa9b027f9"}
Jul 17 09:25:02.958750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:02.958709    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"]
Jul 17 09:25:02.966059 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:02.966034    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"]
Jul 17 09:25:02.966210 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:02.966173    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"
Jul 17 09:25:02.969450 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:02.969429    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"
Jul 17 09:25:02.996549 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:02.996523    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-2mcvp\""
Jul 17 09:25:02.996659 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:02.996603    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 17 09:25:02.996659 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:02.996640    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 17 09:25:02.997718 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:02.997698    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vslxc\""
Jul 17 09:25:03.045946 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.045911    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"]
Jul 17 09:25:03.046226 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.046196    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wwmw\" (UniqueName: \"kubernetes.io/projected/9451d4ab-821c-4361-8f98-963d0f02f5b1-kube-api-access-6wwmw\") pod \"llmisvc-controller-manager-777c6bd7d5-mh7dz\" (UID: \"9451d4ab-821c-4361-8f98-963d0f02f5b1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"
Jul 17 09:25:03.046332 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.046246    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9763398-f47f-4a5e-9048-776d3b931e4f-cert\") pod \"kserve-controller-manager-d6f4b65b8-j6d95\" (UID: \"e9763398-f47f-4a5e-9048-776d3b931e4f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"
Jul 17 09:25:03.046384 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.046331    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28ft2\" (UniqueName: \"kubernetes.io/projected/e9763398-f47f-4a5e-9048-776d3b931e4f-kube-api-access-28ft2\") pod \"kserve-controller-manager-d6f4b65b8-j6d95\" (UID: \"e9763398-f47f-4a5e-9048-776d3b931e4f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"
Jul 17 09:25:03.046433 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.046412    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9451d4ab-821c-4361-8f98-963d0f02f5b1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mh7dz\" (UID: \"9451d4ab-821c-4361-8f98-963d0f02f5b1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"
Jul 17 09:25:03.047928 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.047905    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"]
Jul 17 09:25:03.147009 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.146965    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wwmw\" (UniqueName: \"kubernetes.io/projected/9451d4ab-821c-4361-8f98-963d0f02f5b1-kube-api-access-6wwmw\") pod \"llmisvc-controller-manager-777c6bd7d5-mh7dz\" (UID: \"9451d4ab-821c-4361-8f98-963d0f02f5b1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"
Jul 17 09:25:03.147187 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.147025    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9763398-f47f-4a5e-9048-776d3b931e4f-cert\") pod \"kserve-controller-manager-d6f4b65b8-j6d95\" (UID: \"e9763398-f47f-4a5e-9048-776d3b931e4f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"
Jul 17 09:25:03.147187 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.147078    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28ft2\" (UniqueName: \"kubernetes.io/projected/e9763398-f47f-4a5e-9048-776d3b931e4f-kube-api-access-28ft2\") pod \"kserve-controller-manager-d6f4b65b8-j6d95\" (UID: \"e9763398-f47f-4a5e-9048-776d3b931e4f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"
Jul 17 09:25:03.147187 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.147141    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9451d4ab-821c-4361-8f98-963d0f02f5b1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mh7dz\" (UID: \"9451d4ab-821c-4361-8f98-963d0f02f5b1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"
Jul 17 09:25:03.149821 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.149800    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9763398-f47f-4a5e-9048-776d3b931e4f-cert\") pod \"kserve-controller-manager-d6f4b65b8-j6d95\" (UID: \"e9763398-f47f-4a5e-9048-776d3b931e4f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"
Jul 17 09:25:03.149955 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.149891    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9451d4ab-821c-4361-8f98-963d0f02f5b1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mh7dz\" (UID: \"9451d4ab-821c-4361-8f98-963d0f02f5b1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"
Jul 17 09:25:03.173542 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.173514    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wwmw\" (UniqueName: \"kubernetes.io/projected/9451d4ab-821c-4361-8f98-963d0f02f5b1-kube-api-access-6wwmw\") pod \"llmisvc-controller-manager-777c6bd7d5-mh7dz\" (UID: \"9451d4ab-821c-4361-8f98-963d0f02f5b1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"
Jul 17 09:25:03.189070 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.189047    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28ft2\" (UniqueName: \"kubernetes.io/projected/e9763398-f47f-4a5e-9048-776d3b931e4f-kube-api-access-28ft2\") pod \"kserve-controller-manager-d6f4b65b8-j6d95\" (UID: \"e9763398-f47f-4a5e-9048-776d3b931e4f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"
Jul 17 09:25:03.277437 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.277398    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 17 09:25:03.277529 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.277486    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 17 09:25:03.281205 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.281187    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"
Jul 17 09:25:03.288408 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.288390    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"
Jul 17 09:25:03.462922 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.462889    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn" event={"ID":"04aa59a4-2be5-498f-b3e6-6c1d9bf92551","Type":"ContainerStarted","Data":"71b1a2e211dcc07422aec4ba98e7989e36e78995f93b837b9238217327c71052"}
Jul 17 09:25:03.463106 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.462945    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:03.491765 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.491736    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"]
Jul 17 09:25:03.494754 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:25:03.494731    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9451d4ab_821c_4361_8f98_963d0f02f5b1.slice/crio-e4b2d5ca13d53f7e3c971961516615538e11867fa30fe0fbe154d4709f43c601 WatchSource:0}: Error finding container e4b2d5ca13d53f7e3c971961516615538e11867fa30fe0fbe154d4709f43c601: Status 404 returned error can't find the container with id e4b2d5ca13d53f7e3c971961516615538e11867fa30fe0fbe154d4709f43c601
Jul 17 09:25:03.500902 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.500866    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"]
Jul 17 09:25:03.502902 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:25:03.502876    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9763398_f47f_4a5e_9048_776d3b931e4f.slice/crio-5c09605ed7745c4c6a0b84ccbb0e8cbcccaa7eb1eac39ca8e41341ae2a432053 WatchSource:0}: Error finding container 5c09605ed7745c4c6a0b84ccbb0e8cbcccaa7eb1eac39ca8e41341ae2a432053: Status 404 returned error can't find the container with id 5c09605ed7745c4c6a0b84ccbb0e8cbcccaa7eb1eac39ca8e41341ae2a432053
Jul 17 09:25:03.525271 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:03.525188    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn" podStartSLOduration=0.843427207 podStartE2EDuration="3.52516928s" totalImagesPullingTime="2.681742073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:25:03.524836288 +0000 UTC m=+387.589657013" watchObservedRunningTime="2026-07-17 09:25:03.52516928 +0000 UTC m=+387.589990004"
Jul 17 09:25:04.414021 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:04.413985    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-jgsgk"
Jul 17 09:25:04.470946 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:04.470908    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz" event={"ID":"9451d4ab-821c-4361-8f98-963d0f02f5b1","Type":"ContainerStarted","Data":"e4b2d5ca13d53f7e3c971961516615538e11867fa30fe0fbe154d4709f43c601"}
Jul 17 09:25:04.472927 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:04.472860    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95" event={"ID":"e9763398-f47f-4a5e-9048-776d3b931e4f","Type":"ContainerStarted","Data":"5c09605ed7745c4c6a0b84ccbb0e8cbcccaa7eb1eac39ca8e41341ae2a432053"}
Jul 17 09:25:04.474935 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:04.474903    2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-ktvxn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 09:25:04.475057 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:04.474965    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn" podUID="04aa59a4-2be5-498f-b3e6-6c1d9bf92551" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 09:25:06.427249 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:06.427202    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-g6z4w"
Jul 17 09:25:06.487098 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:06.487060    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95" event={"ID":"e9763398-f47f-4a5e-9048-776d3b931e4f","Type":"ContainerStarted","Data":"e9fc0892cd27937ae000fc4f63303ebe82a6a8703e690a7735f1ca1e703a7aae"}
Jul 17 09:25:06.487273 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:06.487221    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"
Jul 17 09:25:06.604442 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:06.604382    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95" podStartSLOduration=2.092687339 podStartE2EDuration="4.604364733s" totalImagesPullingTime="2.511677394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:25:06.594708554 +0000 UTC m=+390.659529278" watchObservedRunningTime="2026-07-17 09:25:06.604364733 +0000 UTC m=+390.669185454"
Jul 17 09:25:07.474055 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:07.474019    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ktvxn"
Jul 17 09:25:08.500250 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:08.500209    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz" event={"ID":"9451d4ab-821c-4361-8f98-963d0f02f5b1","Type":"ContainerStarted","Data":"9f7603db225000ecbab9b3b67ff7a5b4b1287259aab88b7165aec02e322f5dc2"}
Jul 17 09:25:08.500752 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:08.500345    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"
Jul 17 09:25:08.523179 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:08.523119    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz" podStartSLOduration=2.194439675 podStartE2EDuration="6.523103805s" totalImagesPullingTime="4.32866413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:25:08.521147631 +0000 UTC m=+392.585968355" watchObservedRunningTime="2026-07-17 09:25:08.523103805 +0000 UTC m=+392.587924527"
Jul 17 09:25:37.498744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:37.498713    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-j6d95"
Jul 17 09:25:39.506770 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:25:39.506737    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mh7dz"
Jul 17 09:26:07.166401 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.166322    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"]
Jul 17 09:26:07.168938 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.168921    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:07.171654 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.171628    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 09:26:07.171789 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.171697    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 09:26:07.172560 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.172546    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xwls4\""
Jul 17 09:26:07.181819 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.181797    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"]
Jul 17 09:26:07.218993 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.218952    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:07.219166 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.219009    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:07.219166 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.219095    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js9hs\" (UniqueName: \"kubernetes.io/projected/352624fd-81f2-430b-9ba9-a413d64c108a-kube-api-access-js9hs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:07.319960 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.319919    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js9hs\" (UniqueName: \"kubernetes.io/projected/352624fd-81f2-430b-9ba9-a413d64c108a-kube-api-access-js9hs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:07.320159 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.320027    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:07.320159 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.320064    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:07.320512 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.320488    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:07.320512 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.320503    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:07.330636 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.330616    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js9hs\" (UniqueName: \"kubernetes.io/projected/352624fd-81f2-430b-9ba9-a413d64c108a-kube-api-access-js9hs\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:07.478599 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.478563    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:07.537032 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.536997    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"]
Jul 17 09:26:07.540787 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.540765    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:07.549221 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.549196    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"]
Jul 17 09:26:07.622842 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.622815    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"]
Jul 17 09:26:07.623168 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.623142    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4rsc\" (UniqueName: \"kubernetes.io/projected/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-kube-api-access-g4rsc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:07.623223 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.623205    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:07.623299 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.623285    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:07.625147 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:26:07.625117    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod352624fd_81f2_430b_9ba9_a413d64c108a.slice/crio-77f512035df66733d079c256d252c8a023df4766deb9e273a27c36a24469c266 WatchSource:0}: Error finding container 77f512035df66733d079c256d252c8a023df4766deb9e273a27c36a24469c266: Status 404 returned error can't find the container with id 77f512035df66733d079c256d252c8a023df4766deb9e273a27c36a24469c266
Jul 17 09:26:07.724352 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.724323    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4rsc\" (UniqueName: \"kubernetes.io/projected/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-kube-api-access-g4rsc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:07.724497 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.724365    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:07.724497 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.724420    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:07.724804 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.724783    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:07.724841 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.724801    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:07.733882 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.733825    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4rsc\" (UniqueName: \"kubernetes.io/projected/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-kube-api-access-g4rsc\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:07.789524 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.789490    2571 generic.go:356] "Generic (PLEG): container finished" podID="352624fd-81f2-430b-9ba9-a413d64c108a" containerID="9ccd1c10572ae67e182cc64debe9b0c4e892796bf9f549f94720646cc9b367f5" exitCode=0
Jul 17 09:26:07.789685 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.789532    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2" event={"ID":"352624fd-81f2-430b-9ba9-a413d64c108a","Type":"ContainerDied","Data":"9ccd1c10572ae67e182cc64debe9b0c4e892796bf9f549f94720646cc9b367f5"}
Jul 17 09:26:07.789685 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.789588    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2" event={"ID":"352624fd-81f2-430b-9ba9-a413d64c108a","Type":"ContainerStarted","Data":"77f512035df66733d079c256d252c8a023df4766deb9e273a27c36a24469c266"}
Jul 17 09:26:07.855059 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.855030    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:07.930959 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.930925    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"]
Jul 17 09:26:07.934287 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.934271    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:07.945178 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.945144    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"]
Jul 17 09:26:07.991255 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:07.991159    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"]
Jul 17 09:26:07.994001 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:26:07.993974    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e8308e7_7b0c_4b8c_988e_b4ffc7a61416.slice/crio-bb3f80ed0a22e12518d21c0e9cce9568f36b58776d35dad9daa5dfbc437c956b WatchSource:0}: Error finding container bb3f80ed0a22e12518d21c0e9cce9568f36b58776d35dad9daa5dfbc437c956b: Status 404 returned error can't find the container with id bb3f80ed0a22e12518d21c0e9cce9568f36b58776d35dad9daa5dfbc437c956b
Jul 17 09:26:08.027355 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.027329    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjpp4\" (UniqueName: \"kubernetes.io/projected/a853521b-dc4b-458c-8259-2d4bc3692f97-kube-api-access-fjpp4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:08.027455 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.027428    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:08.027522 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.027504    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:08.128517 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.128453    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:08.128667 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.128535    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjpp4\" (UniqueName: \"kubernetes.io/projected/a853521b-dc4b-458c-8259-2d4bc3692f97-kube-api-access-fjpp4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:08.128667 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.128620    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:08.128898 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.128875    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:08.128974 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.128956    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:08.141385 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.141356    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjpp4\" (UniqueName: \"kubernetes.io/projected/a853521b-dc4b-458c-8259-2d4bc3692f97-kube-api-access-fjpp4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:08.247110 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.247030    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:08.325076 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.325036    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"]
Jul 17 09:26:08.328113 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.328047    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:08.339844 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.339713    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"]
Jul 17 09:26:08.387529 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.387495    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"]
Jul 17 09:26:08.390685 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:26:08.390663    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda853521b_dc4b_458c_8259_2d4bc3692f97.slice/crio-47423dc863f8e3d2879bbeaa54876cdcc32416bd6f0d40f6fc15095e30558e74 WatchSource:0}: Error finding container 47423dc863f8e3d2879bbeaa54876cdcc32416bd6f0d40f6fc15095e30558e74: Status 404 returned error can't find the container with id 47423dc863f8e3d2879bbeaa54876cdcc32416bd6f0d40f6fc15095e30558e74
Jul 17 09:26:08.432694 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.432652    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdr4w\" (UniqueName: \"kubernetes.io/projected/c22918ea-3e5c-4cf0-997a-60924541367c-kube-api-access-qdr4w\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:08.432853 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.432790    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:08.432853 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.432814    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:08.533931 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.533886    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:08.533931 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.533933    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:08.534215 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.534026    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdr4w\" (UniqueName: \"kubernetes.io/projected/c22918ea-3e5c-4cf0-997a-60924541367c-kube-api-access-qdr4w\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:08.534317 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.534296    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:08.534382 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.534353    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:08.543840 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.543804    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdr4w\" (UniqueName: \"kubernetes.io/projected/c22918ea-3e5c-4cf0-997a-60924541367c-kube-api-access-qdr4w\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:08.663157 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.663117    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:08.796909 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.796685    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn" event={"ID":"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416","Type":"ContainerDied","Data":"02295a7f5ebd1de24c13f93523cb855915d734876f276b159f8a5b1bacfb47cc"}
Jul 17 09:26:08.796909 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.796637    2571 generic.go:356] "Generic (PLEG): container finished" podID="5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" containerID="02295a7f5ebd1de24c13f93523cb855915d734876f276b159f8a5b1bacfb47cc" exitCode=0
Jul 17 09:26:08.797115 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.796929    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn" event={"ID":"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416","Type":"ContainerStarted","Data":"bb3f80ed0a22e12518d21c0e9cce9568f36b58776d35dad9daa5dfbc437c956b"}
Jul 17 09:26:08.799165 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.799137    2571 generic.go:356] "Generic (PLEG): container finished" podID="352624fd-81f2-430b-9ba9-a413d64c108a" containerID="35a5716c1681b4078e6baa0fb3c660fe6660727b27d600d1dd7eb68c4f7f4119" exitCode=0
Jul 17 09:26:08.799263 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.799221    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2" event={"ID":"352624fd-81f2-430b-9ba9-a413d64c108a","Type":"ContainerDied","Data":"35a5716c1681b4078e6baa0fb3c660fe6660727b27d600d1dd7eb68c4f7f4119"}
Jul 17 09:26:08.799915 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.799804    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"]
Jul 17 09:26:08.801128 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.800930    2571 generic.go:356] "Generic (PLEG): container finished" podID="a853521b-dc4b-458c-8259-2d4bc3692f97" containerID="6dc264b907a1f16305adc6748f80dcce930ee3c77f4443f158ba888957f9dcd0" exitCode=0
Jul 17 09:26:08.801128 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.801015    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf" event={"ID":"a853521b-dc4b-458c-8259-2d4bc3692f97","Type":"ContainerDied","Data":"6dc264b907a1f16305adc6748f80dcce930ee3c77f4443f158ba888957f9dcd0"}
Jul 17 09:26:08.801128 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:08.801057    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf" event={"ID":"a853521b-dc4b-458c-8259-2d4bc3692f97","Type":"ContainerStarted","Data":"47423dc863f8e3d2879bbeaa54876cdcc32416bd6f0d40f6fc15095e30558e74"}
Jul 17 09:26:08.801436 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:26:08.801408    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22918ea_3e5c_4cf0_997a_60924541367c.slice/crio-114a8e61a7451fddb2246f4504af523b91200669b7ae0fb17ad9fff69ab7c93d WatchSource:0}: Error finding container 114a8e61a7451fddb2246f4504af523b91200669b7ae0fb17ad9fff69ab7c93d: Status 404 returned error can't find the container with id 114a8e61a7451fddb2246f4504af523b91200669b7ae0fb17ad9fff69ab7c93d
Jul 17 09:26:09.811037 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:09.810990    2571 generic.go:356] "Generic (PLEG): container finished" podID="352624fd-81f2-430b-9ba9-a413d64c108a" containerID="85be5e94c2f2beb7b722be98e685ca6bf9ef4584dea2d2ecf9be9cf840ca44f1" exitCode=0
Jul 17 09:26:09.811438 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:09.811055    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2" event={"ID":"352624fd-81f2-430b-9ba9-a413d64c108a","Type":"ContainerDied","Data":"85be5e94c2f2beb7b722be98e685ca6bf9ef4584dea2d2ecf9be9cf840ca44f1"}
Jul 17 09:26:09.813646 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:09.813610    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf" event={"ID":"a853521b-dc4b-458c-8259-2d4bc3692f97","Type":"ContainerStarted","Data":"997606ed28485c864ea79af63d821068637c34189468047285179c5cc475bcae"}
Jul 17 09:26:09.815423 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:09.815401    2571 generic.go:356] "Generic (PLEG): container finished" podID="c22918ea-3e5c-4cf0-997a-60924541367c" containerID="0ebdb5be64c00df1753da5bb35928f91c5aa2a6973ae0e911d50d8c5e258d093" exitCode=0
Jul 17 09:26:09.815555 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:09.815493    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq" event={"ID":"c22918ea-3e5c-4cf0-997a-60924541367c","Type":"ContainerDied","Data":"0ebdb5be64c00df1753da5bb35928f91c5aa2a6973ae0e911d50d8c5e258d093"}
Jul 17 09:26:09.815555 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:09.815535    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq" event={"ID":"c22918ea-3e5c-4cf0-997a-60924541367c","Type":"ContainerStarted","Data":"114a8e61a7451fddb2246f4504af523b91200669b7ae0fb17ad9fff69ab7c93d"}
Jul 17 09:26:09.818300 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:09.818279    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn" event={"ID":"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416","Type":"ContainerStarted","Data":"08fd24dbde00e81bf747c3d4c28d3d4c0e64716e7a6aea2d7f7a5cf7e13e4186"}
Jul 17 09:26:10.831262 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:10.831224    2571 generic.go:356] "Generic (PLEG): container finished" podID="5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" containerID="08fd24dbde00e81bf747c3d4c28d3d4c0e64716e7a6aea2d7f7a5cf7e13e4186" exitCode=0
Jul 17 09:26:10.831729 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:10.831312    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn" event={"ID":"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416","Type":"ContainerDied","Data":"08fd24dbde00e81bf747c3d4c28d3d4c0e64716e7a6aea2d7f7a5cf7e13e4186"}
Jul 17 09:26:10.833365 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:10.833342    2571 generic.go:356] "Generic (PLEG): container finished" podID="a853521b-dc4b-458c-8259-2d4bc3692f97" containerID="997606ed28485c864ea79af63d821068637c34189468047285179c5cc475bcae" exitCode=0
Jul 17 09:26:10.833504 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:10.833433    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf" event={"ID":"a853521b-dc4b-458c-8259-2d4bc3692f97","Type":"ContainerDied","Data":"997606ed28485c864ea79af63d821068637c34189468047285179c5cc475bcae"}
Jul 17 09:26:10.835343 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:10.835272    2571 generic.go:356] "Generic (PLEG): container finished" podID="c22918ea-3e5c-4cf0-997a-60924541367c" containerID="5bb94c1b402455590565c195b0a6bd38914b67d967b1c0a8123141fde9293948" exitCode=0
Jul 17 09:26:10.835414 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:10.835347    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq" event={"ID":"c22918ea-3e5c-4cf0-997a-60924541367c","Type":"ContainerDied","Data":"5bb94c1b402455590565c195b0a6bd38914b67d967b1c0a8123141fde9293948"}
Jul 17 09:26:10.988608 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:10.988583    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:11.059022 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.058996    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-js9hs\" (UniqueName: \"kubernetes.io/projected/352624fd-81f2-430b-9ba9-a413d64c108a-kube-api-access-js9hs\") pod \"352624fd-81f2-430b-9ba9-a413d64c108a\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") "
Jul 17 09:26:11.059121 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.059088    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-util\") pod \"352624fd-81f2-430b-9ba9-a413d64c108a\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") "
Jul 17 09:26:11.059156 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.059126    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-bundle\") pod \"352624fd-81f2-430b-9ba9-a413d64c108a\" (UID: \"352624fd-81f2-430b-9ba9-a413d64c108a\") "
Jul 17 09:26:11.059683 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.059653    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-bundle" (OuterVolumeSpecName: "bundle") pod "352624fd-81f2-430b-9ba9-a413d64c108a" (UID: "352624fd-81f2-430b-9ba9-a413d64c108a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:26:11.061056 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.061034    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/352624fd-81f2-430b-9ba9-a413d64c108a-kube-api-access-js9hs" (OuterVolumeSpecName: "kube-api-access-js9hs") pod "352624fd-81f2-430b-9ba9-a413d64c108a" (UID: "352624fd-81f2-430b-9ba9-a413d64c108a"). InnerVolumeSpecName "kube-api-access-js9hs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:26:11.064890 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.064867    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-util" (OuterVolumeSpecName: "util") pod "352624fd-81f2-430b-9ba9-a413d64c108a" (UID: "352624fd-81f2-430b-9ba9-a413d64c108a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:26:11.160601 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.160576    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-util\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:11.160601 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.160598    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/352624fd-81f2-430b-9ba9-a413d64c108a-bundle\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:11.160717 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.160609    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js9hs\" (UniqueName: \"kubernetes.io/projected/352624fd-81f2-430b-9ba9-a413d64c108a-kube-api-access-js9hs\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:11.842752 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.842714    2571 generic.go:356] "Generic (PLEG): container finished" podID="5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" containerID="c537d4c0b9c11fc6487e19118c1aae536bae3026e35e910a036d501436ed48d9" exitCode=0
Jul 17 09:26:11.843182 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.842805    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn" event={"ID":"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416","Type":"ContainerDied","Data":"c537d4c0b9c11fc6487e19118c1aae536bae3026e35e910a036d501436ed48d9"}
Jul 17 09:26:11.844596 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.844577    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2"
Jul 17 09:26:11.844596 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.844589    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2" event={"ID":"352624fd-81f2-430b-9ba9-a413d64c108a","Type":"ContainerDied","Data":"77f512035df66733d079c256d252c8a023df4766deb9e273a27c36a24469c266"}
Jul 17 09:26:11.844791 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.844624    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77f512035df66733d079c256d252c8a023df4766deb9e273a27c36a24469c266"
Jul 17 09:26:11.846887 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.846865    2571 generic.go:356] "Generic (PLEG): container finished" podID="a853521b-dc4b-458c-8259-2d4bc3692f97" containerID="e625504484eeac64afad31b59252de39c750f6576d73811c947a20300eb2f236" exitCode=0
Jul 17 09:26:11.846996 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.846939    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf" event={"ID":"a853521b-dc4b-458c-8259-2d4bc3692f97","Type":"ContainerDied","Data":"e625504484eeac64afad31b59252de39c750f6576d73811c947a20300eb2f236"}
Jul 17 09:26:11.848852 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.848833    2571 generic.go:356] "Generic (PLEG): container finished" podID="c22918ea-3e5c-4cf0-997a-60924541367c" containerID="52fa126e04cd942a640cf7784c9aa9b69840decb139243e4b57ba4e66062f46c" exitCode=0
Jul 17 09:26:11.848939 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:11.848904    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq" event={"ID":"c22918ea-3e5c-4cf0-997a-60924541367c","Type":"ContainerDied","Data":"52fa126e04cd942a640cf7784c9aa9b69840decb139243e4b57ba4e66062f46c"}
Jul 17 09:26:13.000407 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.000377    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:13.040505 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.040482    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:13.043581 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.043563    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:13.078178 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.078149    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-util\") pod \"a853521b-dc4b-458c-8259-2d4bc3692f97\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") "
Jul 17 09:26:13.078316 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.078225    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjpp4\" (UniqueName: \"kubernetes.io/projected/a853521b-dc4b-458c-8259-2d4bc3692f97-kube-api-access-fjpp4\") pod \"a853521b-dc4b-458c-8259-2d4bc3692f97\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") "
Jul 17 09:26:13.078316 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.078296    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-bundle\") pod \"a853521b-dc4b-458c-8259-2d4bc3692f97\" (UID: \"a853521b-dc4b-458c-8259-2d4bc3692f97\") "
Jul 17 09:26:13.078887 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.078859    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-bundle" (OuterVolumeSpecName: "bundle") pod "a853521b-dc4b-458c-8259-2d4bc3692f97" (UID: "a853521b-dc4b-458c-8259-2d4bc3692f97"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:26:13.081265 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.081229    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a853521b-dc4b-458c-8259-2d4bc3692f97-kube-api-access-fjpp4" (OuterVolumeSpecName: "kube-api-access-fjpp4") pod "a853521b-dc4b-458c-8259-2d4bc3692f97" (UID: "a853521b-dc4b-458c-8259-2d4bc3692f97"). InnerVolumeSpecName "kube-api-access-fjpp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:26:13.083962 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.083929    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-util" (OuterVolumeSpecName: "util") pod "a853521b-dc4b-458c-8259-2d4bc3692f97" (UID: "a853521b-dc4b-458c-8259-2d4bc3692f97"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:26:13.179742 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.179663    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdr4w\" (UniqueName: \"kubernetes.io/projected/c22918ea-3e5c-4cf0-997a-60924541367c-kube-api-access-qdr4w\") pod \"c22918ea-3e5c-4cf0-997a-60924541367c\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") "
Jul 17 09:26:13.179894 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.179804    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4rsc\" (UniqueName: \"kubernetes.io/projected/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-kube-api-access-g4rsc\") pod \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") "
Jul 17 09:26:13.179894 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.179846    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-util\") pod \"c22918ea-3e5c-4cf0-997a-60924541367c\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") "
Jul 17 09:26:13.179979 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.179895    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-bundle\") pod \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") "
Jul 17 09:26:13.179979 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.179912    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-util\") pod \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\" (UID: \"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416\") "
Jul 17 09:26:13.179979 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.179951    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-bundle\") pod \"c22918ea-3e5c-4cf0-997a-60924541367c\" (UID: \"c22918ea-3e5c-4cf0-997a-60924541367c\") "
Jul 17 09:26:13.180212 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.180194    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjpp4\" (UniqueName: \"kubernetes.io/projected/a853521b-dc4b-458c-8259-2d4bc3692f97-kube-api-access-fjpp4\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:13.180274 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.180237    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-bundle\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:13.180274 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.180254    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a853521b-dc4b-458c-8259-2d4bc3692f97-util\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:13.180561 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.180520    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-bundle" (OuterVolumeSpecName: "bundle") pod "5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" (UID: "5e8308e7-7b0c-4b8c-988e-b4ffc7a61416"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:26:13.180561 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.180516    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-bundle" (OuterVolumeSpecName: "bundle") pod "c22918ea-3e5c-4cf0-997a-60924541367c" (UID: "c22918ea-3e5c-4cf0-997a-60924541367c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:26:13.182122 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.182096    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c22918ea-3e5c-4cf0-997a-60924541367c-kube-api-access-qdr4w" (OuterVolumeSpecName: "kube-api-access-qdr4w") pod "c22918ea-3e5c-4cf0-997a-60924541367c" (UID: "c22918ea-3e5c-4cf0-997a-60924541367c"). InnerVolumeSpecName "kube-api-access-qdr4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:26:13.182372 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.182351    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-kube-api-access-g4rsc" (OuterVolumeSpecName: "kube-api-access-g4rsc") pod "5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" (UID: "5e8308e7-7b0c-4b8c-988e-b4ffc7a61416"). InnerVolumeSpecName "kube-api-access-g4rsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:26:13.185972 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.185908    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-util" (OuterVolumeSpecName: "util") pod "5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" (UID: "5e8308e7-7b0c-4b8c-988e-b4ffc7a61416"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:26:13.186258 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.186230    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-util" (OuterVolumeSpecName: "util") pod "c22918ea-3e5c-4cf0-997a-60924541367c" (UID: "c22918ea-3e5c-4cf0-997a-60924541367c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:26:13.281173 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.281125    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4rsc\" (UniqueName: \"kubernetes.io/projected/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-kube-api-access-g4rsc\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:13.281173 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.281168    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-util\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:13.281173 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.281180    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-bundle\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:13.281173 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.281187    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e8308e7-7b0c-4b8c-988e-b4ffc7a61416-util\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:13.281439 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.281196    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c22918ea-3e5c-4cf0-997a-60924541367c-bundle\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:13.281439 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.281205    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdr4w\" (UniqueName: \"kubernetes.io/projected/c22918ea-3e5c-4cf0-997a-60924541367c-kube-api-access-qdr4w\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:26:13.861291 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.861267    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf"
Jul 17 09:26:13.861427 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.861264    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf" event={"ID":"a853521b-dc4b-458c-8259-2d4bc3692f97","Type":"ContainerDied","Data":"47423dc863f8e3d2879bbeaa54876cdcc32416bd6f0d40f6fc15095e30558e74"}
Jul 17 09:26:13.861427 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.861422    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47423dc863f8e3d2879bbeaa54876cdcc32416bd6f0d40f6fc15095e30558e74"
Jul 17 09:26:13.863277 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.863241    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq"
Jul 17 09:26:13.863277 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.863268    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq" event={"ID":"c22918ea-3e5c-4cf0-997a-60924541367c","Type":"ContainerDied","Data":"114a8e61a7451fddb2246f4504af523b91200669b7ae0fb17ad9fff69ab7c93d"}
Jul 17 09:26:13.863503 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.863308    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="114a8e61a7451fddb2246f4504af523b91200669b7ae0fb17ad9fff69ab7c93d"
Jul 17 09:26:13.865444 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.865421    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn" event={"ID":"5e8308e7-7b0c-4b8c-988e-b4ffc7a61416","Type":"ContainerDied","Data":"bb3f80ed0a22e12518d21c0e9cce9568f36b58776d35dad9daa5dfbc437c956b"}
Jul 17 09:26:13.865572 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.865458    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb3f80ed0a22e12518d21c0e9cce9568f36b58776d35dad9daa5dfbc437c956b"
Jul 17 09:26:13.865572 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:13.865565    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn"
Jul 17 09:26:31.336300 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336266    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq"]
Jul 17 09:26:31.336660 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336637    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a853521b-dc4b-458c-8259-2d4bc3692f97" containerName="pull"
Jul 17 09:26:31.336660 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336649    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a853521b-dc4b-458c-8259-2d4bc3692f97" containerName="pull"
Jul 17 09:26:31.336660 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336659    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" containerName="extract"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336665    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" containerName="extract"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336673    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22918ea-3e5c-4cf0-997a-60924541367c" containerName="util"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336679    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22918ea-3e5c-4cf0-997a-60924541367c" containerName="util"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336686    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" containerName="pull"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336690    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" containerName="pull"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336703    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="352624fd-81f2-430b-9ba9-a413d64c108a" containerName="pull"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336708    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="352624fd-81f2-430b-9ba9-a413d64c108a" containerName="pull"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336716    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="352624fd-81f2-430b-9ba9-a413d64c108a" containerName="extract"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336721    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="352624fd-81f2-430b-9ba9-a413d64c108a" containerName="extract"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336728    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22918ea-3e5c-4cf0-997a-60924541367c" containerName="extract"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336733    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22918ea-3e5c-4cf0-997a-60924541367c" containerName="extract"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336740    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" containerName="util"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336744    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" containerName="util"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336750    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a853521b-dc4b-458c-8259-2d4bc3692f97" containerName="util"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336755    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a853521b-dc4b-458c-8259-2d4bc3692f97" containerName="util"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336761    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="352624fd-81f2-430b-9ba9-a413d64c108a" containerName="util"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336766    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="352624fd-81f2-430b-9ba9-a413d64c108a" containerName="util"
Jul 17 09:26:31.336766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336773    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c22918ea-3e5c-4cf0-997a-60924541367c" containerName="pull"
Jul 17 09:26:31.337293 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336778    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c22918ea-3e5c-4cf0-997a-60924541367c" containerName="pull"
Jul 17 09:26:31.337293 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336784    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a853521b-dc4b-458c-8259-2d4bc3692f97" containerName="extract"
Jul 17 09:26:31.337293 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336790    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a853521b-dc4b-458c-8259-2d4bc3692f97" containerName="extract"
Jul 17 09:26:31.337293 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336842    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e8308e7-7b0c-4b8c-988e-b4ffc7a61416" containerName="extract"
Jul 17 09:26:31.337293 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336849    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="352624fd-81f2-430b-9ba9-a413d64c108a" containerName="extract"
Jul 17 09:26:31.337293 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336855    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c22918ea-3e5c-4cf0-997a-60924541367c" containerName="extract"
Jul 17 09:26:31.337293 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.336859    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a853521b-dc4b-458c-8259-2d4bc3692f97" containerName="extract"
Jul 17 09:26:31.338607 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.338590    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq"
Jul 17 09:26:31.341552 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.341532    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-cqmlp\""
Jul 17 09:26:31.341709 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.341689    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 09:26:31.342123 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.342106    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 09:26:31.344176 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.344155    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6qxg\" (UniqueName: \"kubernetes.io/projected/88845c50-3a6f-4983-bf5d-0bd429c7cac9-kube-api-access-r6qxg\") pod \"dns-operator-controller-manager-864dd78c4c-kdtdq\" (UID: \"88845c50-3a6f-4983-bf5d-0bd429c7cac9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq"
Jul 17 09:26:31.355296 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.355281    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 17 09:26:31.380202 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.380175    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq"]
Jul 17 09:26:31.444926 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.444885    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6qxg\" (UniqueName: \"kubernetes.io/projected/88845c50-3a6f-4983-bf5d-0bd429c7cac9-kube-api-access-r6qxg\") pod \"dns-operator-controller-manager-864dd78c4c-kdtdq\" (UID: \"88845c50-3a6f-4983-bf5d-0bd429c7cac9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq"
Jul 17 09:26:31.463583 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.463551    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6qxg\" (UniqueName: \"kubernetes.io/projected/88845c50-3a6f-4983-bf5d-0bd429c7cac9-kube-api-access-r6qxg\") pod \"dns-operator-controller-manager-864dd78c4c-kdtdq\" (UID: \"88845c50-3a6f-4983-bf5d-0bd429c7cac9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq"
Jul 17 09:26:31.648270 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.648179    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq"
Jul 17 09:26:31.847154 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.847120    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq"]
Jul 17 09:26:31.848643 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:26:31.848609    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88845c50_3a6f_4983_bf5d_0bd429c7cac9.slice/crio-f2d7fd2aec89341fb44e51a27dc6a7eb564a31a64d1ec10346c1920943661910 WatchSource:0}: Error finding container f2d7fd2aec89341fb44e51a27dc6a7eb564a31a64d1ec10346c1920943661910: Status 404 returned error can't find the container with id f2d7fd2aec89341fb44e51a27dc6a7eb564a31a64d1ec10346c1920943661910
Jul 17 09:26:31.959861 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:31.959818    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq" event={"ID":"88845c50-3a6f-4983-bf5d-0bd429c7cac9","Type":"ContainerStarted","Data":"f2d7fd2aec89341fb44e51a27dc6a7eb564a31a64d1ec10346c1920943661910"}
Jul 17 09:26:33.972569 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:33.972537    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq" event={"ID":"88845c50-3a6f-4983-bf5d-0bd429c7cac9","Type":"ContainerStarted","Data":"25ff2da887a2aceaa579cecd2dd2bfca24941efa16d3eb2d7b42f2d883cf9536"}
Jul 17 09:26:33.972867 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:33.972668    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq"
Jul 17 09:26:34.009496 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:34.005774    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq" podStartSLOduration=0.965107008 podStartE2EDuration="3.005750455s" totalImagesPullingTime="2.040643447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:26:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:26:34.000231057 +0000 UTC m=+478.065051808" watchObservedRunningTime="2026-07-17 09:26:34.005750455 +0000 UTC m=+478.070571178"
Jul 17 09:26:35.068248 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:35.068211    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl"]
Jul 17 09:26:35.071790 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:35.071772    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl"
Jul 17 09:26:35.075655 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:35.075634    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lvbvt\""
Jul 17 09:26:35.083871 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:35.083849    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl"]
Jul 17 09:26:35.174290 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:35.174251    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x49h9\" (UniqueName: \"kubernetes.io/projected/1012a826-2059-4be3-98a9-eec703236571-kube-api-access-x49h9\") pod \"authorino-operator-75cc8d5cb7-hdxzl\" (UID: \"1012a826-2059-4be3-98a9-eec703236571\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl"
Jul 17 09:26:35.275010 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:35.274972    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x49h9\" (UniqueName: \"kubernetes.io/projected/1012a826-2059-4be3-98a9-eec703236571-kube-api-access-x49h9\") pod \"authorino-operator-75cc8d5cb7-hdxzl\" (UID: \"1012a826-2059-4be3-98a9-eec703236571\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl"
Jul 17 09:26:35.286384 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:35.286353    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x49h9\" (UniqueName: \"kubernetes.io/projected/1012a826-2059-4be3-98a9-eec703236571-kube-api-access-x49h9\") pod \"authorino-operator-75cc8d5cb7-hdxzl\" (UID: \"1012a826-2059-4be3-98a9-eec703236571\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl"
Jul 17 09:26:35.382140 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:35.382058    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl"
Jul 17 09:26:35.526006 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:35.525979    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl"]
Jul 17 09:26:35.527738 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:26:35.527709    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1012a826_2059_4be3_98a9_eec703236571.slice/crio-f213378d62e1a9ddc50784df4263d0585c9d5ff1c3f0a2f78e254b647296a0e7 WatchSource:0}: Error finding container f213378d62e1a9ddc50784df4263d0585c9d5ff1c3f0a2f78e254b647296a0e7: Status 404 returned error can't find the container with id f213378d62e1a9ddc50784df4263d0585c9d5ff1c3f0a2f78e254b647296a0e7
Jul 17 09:26:35.983680 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:35.983647    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl" event={"ID":"1012a826-2059-4be3-98a9-eec703236571","Type":"ContainerStarted","Data":"f213378d62e1a9ddc50784df4263d0585c9d5ff1c3f0a2f78e254b647296a0e7"}
Jul 17 09:26:39.006854 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:39.006808    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl" event={"ID":"1012a826-2059-4be3-98a9-eec703236571","Type":"ContainerStarted","Data":"d09b913cf6e0727bf198304e39305f532827ce47b01f2634d509dcb957338e66"}
Jul 17 09:26:39.007263 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:39.006951    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl"
Jul 17 09:26:39.031691 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:39.031638    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl" podStartSLOduration=1.124839432 podStartE2EDuration="4.031620631s" totalImagesPullingTime="2.906781199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:26:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:26:39.029723117 +0000 UTC m=+483.094543840" watchObservedRunningTime="2026-07-17 09:26:39.031620631 +0000 UTC m=+483.096441396"
Jul 17 09:26:44.979690 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:44.979658    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-kdtdq"
Jul 17 09:26:50.015559 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:26:50.015528    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hdxzl"
Jul 17 09:27:20.288103 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.288056    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"]
Jul 17 09:27:20.296165 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.296138    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:20.299318 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.299291    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xwls4\""
Jul 17 09:27:20.299510 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.299387    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 17 09:27:20.301007 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.300982    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"]
Jul 17 09:27:20.384300 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.384263    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"]
Jul 17 09:27:20.386232 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.386207    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/adc632d5-a1f5-40a5-93b4-ad990e23a176-config-file\") pod \"limitador-limitador-64c94fd64f-nlpdw\" (UID: \"adc632d5-a1f5-40a5-93b4-ad990e23a176\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:20.386377 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.386242    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd7nt\" (UniqueName: \"kubernetes.io/projected/adc632d5-a1f5-40a5-93b4-ad990e23a176-kube-api-access-pd7nt\") pod \"limitador-limitador-64c94fd64f-nlpdw\" (UID: \"adc632d5-a1f5-40a5-93b4-ad990e23a176\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:20.487254 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.487211    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/adc632d5-a1f5-40a5-93b4-ad990e23a176-config-file\") pod \"limitador-limitador-64c94fd64f-nlpdw\" (UID: \"adc632d5-a1f5-40a5-93b4-ad990e23a176\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:20.487450 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.487275    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd7nt\" (UniqueName: \"kubernetes.io/projected/adc632d5-a1f5-40a5-93b4-ad990e23a176-kube-api-access-pd7nt\") pod \"limitador-limitador-64c94fd64f-nlpdw\" (UID: \"adc632d5-a1f5-40a5-93b4-ad990e23a176\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:20.487939 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.487914    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/adc632d5-a1f5-40a5-93b4-ad990e23a176-config-file\") pod \"limitador-limitador-64c94fd64f-nlpdw\" (UID: \"adc632d5-a1f5-40a5-93b4-ad990e23a176\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:20.496101 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.496064    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd7nt\" (UniqueName: \"kubernetes.io/projected/adc632d5-a1f5-40a5-93b4-ad990e23a176-kube-api-access-pd7nt\") pod \"limitador-limitador-64c94fd64f-nlpdw\" (UID: \"adc632d5-a1f5-40a5-93b4-ad990e23a176\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:20.609159 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.609072    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:20.749520 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:20.749492    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"]
Jul 17 09:27:20.750881 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:27:20.750855    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadc632d5_a1f5_40a5_93b4_ad990e23a176.slice/crio-f078cf8d5f570ea39c7413da9b98e529eda34ec1e544dda16015203f9942f940 WatchSource:0}: Error finding container f078cf8d5f570ea39c7413da9b98e529eda34ec1e544dda16015203f9942f940: Status 404 returned error can't find the container with id f078cf8d5f570ea39c7413da9b98e529eda34ec1e544dda16015203f9942f940
Jul 17 09:27:21.237385 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:21.237343    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw" event={"ID":"adc632d5-a1f5-40a5-93b4-ad990e23a176","Type":"ContainerStarted","Data":"f078cf8d5f570ea39c7413da9b98e529eda34ec1e544dda16015203f9942f940"}
Jul 17 09:27:24.257199 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:24.257157    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw" event={"ID":"adc632d5-a1f5-40a5-93b4-ad990e23a176","Type":"ContainerStarted","Data":"cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571"}
Jul 17 09:27:24.257607 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:24.257222    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:35.263377 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:35.263279    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:35.264424 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:35.264261    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw" podStartSLOduration=12.339351042 podStartE2EDuration="15.264243503s" totalImagesPullingTime="2.924892461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:24.295393755 +0000 UTC m=+528.360214477" watchObservedRunningTime="2026-07-17 09:27:35.264243503 +0000 UTC m=+539.329064227"
Jul 17 09:27:35.266732 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:35.266708    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"]
Jul 17 09:27:35.316353 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:35.316288    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw" podUID="adc632d5-a1f5-40a5-93b4-ad990e23a176" containerName="limitador" containerID="cri-o://cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571" gracePeriod=30
Jul 17 09:27:35.872156 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:35.872096    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:35.925690 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:35.925655    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/adc632d5-a1f5-40a5-93b4-ad990e23a176-config-file\") pod \"adc632d5-a1f5-40a5-93b4-ad990e23a176\" (UID: \"adc632d5-a1f5-40a5-93b4-ad990e23a176\") "
Jul 17 09:27:35.925900 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:35.925732    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd7nt\" (UniqueName: \"kubernetes.io/projected/adc632d5-a1f5-40a5-93b4-ad990e23a176-kube-api-access-pd7nt\") pod \"adc632d5-a1f5-40a5-93b4-ad990e23a176\" (UID: \"adc632d5-a1f5-40a5-93b4-ad990e23a176\") "
Jul 17 09:27:35.926082 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:35.926061    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adc632d5-a1f5-40a5-93b4-ad990e23a176-config-file" (OuterVolumeSpecName: "config-file") pod "adc632d5-a1f5-40a5-93b4-ad990e23a176" (UID: "adc632d5-a1f5-40a5-93b4-ad990e23a176"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:27:35.928205 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:35.928174    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adc632d5-a1f5-40a5-93b4-ad990e23a176-kube-api-access-pd7nt" (OuterVolumeSpecName: "kube-api-access-pd7nt") pod "adc632d5-a1f5-40a5-93b4-ad990e23a176" (UID: "adc632d5-a1f5-40a5-93b4-ad990e23a176"). InnerVolumeSpecName "kube-api-access-pd7nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:27:36.026826 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.026787    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd7nt\" (UniqueName: \"kubernetes.io/projected/adc632d5-a1f5-40a5-93b4-ad990e23a176-kube-api-access-pd7nt\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:27:36.026826 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.026820    2571 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/adc632d5-a1f5-40a5-93b4-ad990e23a176-config-file\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:27:36.323450 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.323412    2571 generic.go:356] "Generic (PLEG): container finished" podID="adc632d5-a1f5-40a5-93b4-ad990e23a176" containerID="cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571" exitCode=0
Jul 17 09:27:36.323929 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.323477    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"
Jul 17 09:27:36.323929 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.323488    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw" event={"ID":"adc632d5-a1f5-40a5-93b4-ad990e23a176","Type":"ContainerDied","Data":"cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571"}
Jul 17 09:27:36.323929 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.323542    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nlpdw" event={"ID":"adc632d5-a1f5-40a5-93b4-ad990e23a176","Type":"ContainerDied","Data":"f078cf8d5f570ea39c7413da9b98e529eda34ec1e544dda16015203f9942f940"}
Jul 17 09:27:36.323929 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.323564    2571 scope.go:117] "RemoveContainer" containerID="cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571"
Jul 17 09:27:36.333107 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.333087    2571 scope.go:117] "RemoveContainer" containerID="cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571"
Jul 17 09:27:36.333367 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:27:36.333349    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571\": container with ID starting with cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571 not found: ID does not exist" containerID="cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571"
Jul 17 09:27:36.333418 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.333395    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571"} err="failed to get container status \"cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571\": rpc error: code = NotFound desc = could not find container \"cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571\": container with ID starting with cb806b6df1796f186198f906b37a6a5a2891aa317234730ae8efa9df783ba571 not found: ID does not exist"
Jul 17 09:27:36.352762 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.352732    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"]
Jul 17 09:27:36.367542 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.367506    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nlpdw"]
Jul 17 09:27:36.860768 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.860735    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"]
Jul 17 09:27:36.861140 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.861127    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adc632d5-a1f5-40a5-93b4-ad990e23a176" containerName="limitador"
Jul 17 09:27:36.861140 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.861141    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="adc632d5-a1f5-40a5-93b4-ad990e23a176" containerName="limitador"
Jul 17 09:27:36.861232 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.861199    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="adc632d5-a1f5-40a5-93b4-ad990e23a176" containerName="limitador"
Jul 17 09:27:36.865453 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.865433    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"
Jul 17 09:27:36.868899 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.868876    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-f847p\""
Jul 17 09:27:36.869038 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.869019    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 09:27:36.869841 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.869826    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 09:27:36.869901 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.869864    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 17 09:27:36.875198 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.875175    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"]
Jul 17 09:27:36.935613 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.935582    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b585bb07-1d6e-4cff-bb0b-7b77d91f3e73-data\") pod \"postgres-69fb4594cf-x59b8\" (UID: \"b585bb07-1d6e-4cff-bb0b-7b77d91f3e73\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"
Jul 17 09:27:36.935788 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:36.935722    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jrws\" (UniqueName: \"kubernetes.io/projected/b585bb07-1d6e-4cff-bb0b-7b77d91f3e73-kube-api-access-2jrws\") pod \"postgres-69fb4594cf-x59b8\" (UID: \"b585bb07-1d6e-4cff-bb0b-7b77d91f3e73\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"
Jul 17 09:27:37.037157 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:37.037116    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jrws\" (UniqueName: \"kubernetes.io/projected/b585bb07-1d6e-4cff-bb0b-7b77d91f3e73-kube-api-access-2jrws\") pod \"postgres-69fb4594cf-x59b8\" (UID: \"b585bb07-1d6e-4cff-bb0b-7b77d91f3e73\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"
Jul 17 09:27:37.037333 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:37.037175    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b585bb07-1d6e-4cff-bb0b-7b77d91f3e73-data\") pod \"postgres-69fb4594cf-x59b8\" (UID: \"b585bb07-1d6e-4cff-bb0b-7b77d91f3e73\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"
Jul 17 09:27:37.037562 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:37.037543    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b585bb07-1d6e-4cff-bb0b-7b77d91f3e73-data\") pod \"postgres-69fb4594cf-x59b8\" (UID: \"b585bb07-1d6e-4cff-bb0b-7b77d91f3e73\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"
Jul 17 09:27:37.046997 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:37.046973    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jrws\" (UniqueName: \"kubernetes.io/projected/b585bb07-1d6e-4cff-bb0b-7b77d91f3e73-kube-api-access-2jrws\") pod \"postgres-69fb4594cf-x59b8\" (UID: \"b585bb07-1d6e-4cff-bb0b-7b77d91f3e73\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"
Jul 17 09:27:37.176841 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:37.176748    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"
Jul 17 09:27:37.334047 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:37.334022    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"]
Jul 17 09:27:37.335703 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:27:37.335675    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb585bb07_1d6e_4cff_bb0b_7b77d91f3e73.slice/crio-f8d34b225f860bc06a363ffa20aa629d9cd69bf6cf585851be9ba1abf2d48bd2 WatchSource:0}: Error finding container f8d34b225f860bc06a363ffa20aa629d9cd69bf6cf585851be9ba1abf2d48bd2: Status 404 returned error can't find the container with id f8d34b225f860bc06a363ffa20aa629d9cd69bf6cf585851be9ba1abf2d48bd2
Jul 17 09:27:37.759410 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:37.759378    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adc632d5-a1f5-40a5-93b4-ad990e23a176" path="/var/lib/kubelet/pods/adc632d5-a1f5-40a5-93b4-ad990e23a176/volumes"
Jul 17 09:27:38.338257 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:38.338167    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8" event={"ID":"b585bb07-1d6e-4cff-bb0b-7b77d91f3e73","Type":"ContainerStarted","Data":"f8d34b225f860bc06a363ffa20aa629d9cd69bf6cf585851be9ba1abf2d48bd2"}
Jul 17 09:27:42.259892 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:42.259872    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 17 09:27:42.369960 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:42.369924    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8" event={"ID":"b585bb07-1d6e-4cff-bb0b-7b77d91f3e73","Type":"ContainerStarted","Data":"fc432e6ae12a067babcc210b7a942d2e46af54d701d9c4a464c29d9a8c9da003"}
Jul 17 09:27:42.370129 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:42.370035    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"
Jul 17 09:27:48.406112 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:48.406083    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8"
Jul 17 09:27:48.426564 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:48.426516    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-x59b8" podStartSLOduration=7.506788859 podStartE2EDuration="12.426501911s" totalImagesPullingTime="4.919713052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:42.404770302 +0000 UTC m=+546.469591023" watchObservedRunningTime="2026-07-17 09:27:48.426501911 +0000 UTC m=+552.491322634"
Jul 17 09:27:55.581972 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.581933    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8548446449-chlpz"]
Jul 17 09:27:55.605578 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.605546    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8548446449-chlpz"]
Jul 17 09:27:55.605726 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.605671    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:27:55.609317 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.609297    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 09:27:55.610724 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.610704    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-52cp9\""
Jul 17 09:27:55.614148 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.614125    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 09:27:55.700711 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.700675    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmnwc\" (UniqueName: \"kubernetes.io/projected/15c733f4-b43a-4a69-b443-306a214616da-kube-api-access-wmnwc\") pod \"maas-controller-8548446449-chlpz\" (UID: \"15c733f4-b43a-4a69-b443-306a214616da\") " pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:27:55.700873 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.700727    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15c733f4-b43a-4a69-b443-306a214616da-webhook-cert\") pod \"maas-controller-8548446449-chlpz\" (UID: \"15c733f4-b43a-4a69-b443-306a214616da\") " pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:27:55.801427 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.801396    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmnwc\" (UniqueName: \"kubernetes.io/projected/15c733f4-b43a-4a69-b443-306a214616da-kube-api-access-wmnwc\") pod \"maas-controller-8548446449-chlpz\" (UID: \"15c733f4-b43a-4a69-b443-306a214616da\") " pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:27:55.801596 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.801447    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15c733f4-b43a-4a69-b443-306a214616da-webhook-cert\") pod \"maas-controller-8548446449-chlpz\" (UID: \"15c733f4-b43a-4a69-b443-306a214616da\") " pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:27:55.803898 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.803870    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15c733f4-b43a-4a69-b443-306a214616da-webhook-cert\") pod \"maas-controller-8548446449-chlpz\" (UID: \"15c733f4-b43a-4a69-b443-306a214616da\") " pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:27:55.819047 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.819022    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-jsm54"]
Jul 17 09:27:55.823234 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.823197    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:27:55.829326 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.829291    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmnwc\" (UniqueName: \"kubernetes.io/projected/15c733f4-b43a-4a69-b443-306a214616da-kube-api-access-wmnwc\") pod \"maas-controller-8548446449-chlpz\" (UID: \"15c733f4-b43a-4a69-b443-306a214616da\") " pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:27:55.842978 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.842957    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-jsm54"]
Jul 17 09:27:55.902818 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.902786    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a756caa-fee8-436f-ba39-fca2a13763fb-webhook-cert\") pod \"maas-controller-769d7b9cbc-jsm54\" (UID: \"5a756caa-fee8-436f-ba39-fca2a13763fb\") " pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:27:55.902947 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.902844    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66zwh\" (UniqueName: \"kubernetes.io/projected/5a756caa-fee8-436f-ba39-fca2a13763fb-kube-api-access-66zwh\") pod \"maas-controller-769d7b9cbc-jsm54\" (UID: \"5a756caa-fee8-436f-ba39-fca2a13763fb\") " pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:27:55.916690 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:55.916664    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:27:56.004024 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:56.003996    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66zwh\" (UniqueName: \"kubernetes.io/projected/5a756caa-fee8-436f-ba39-fca2a13763fb-kube-api-access-66zwh\") pod \"maas-controller-769d7b9cbc-jsm54\" (UID: \"5a756caa-fee8-436f-ba39-fca2a13763fb\") " pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:27:56.004171 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:56.004098    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a756caa-fee8-436f-ba39-fca2a13763fb-webhook-cert\") pod \"maas-controller-769d7b9cbc-jsm54\" (UID: \"5a756caa-fee8-436f-ba39-fca2a13763fb\") " pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:27:56.006653 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:56.006627    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a756caa-fee8-436f-ba39-fca2a13763fb-webhook-cert\") pod \"maas-controller-769d7b9cbc-jsm54\" (UID: \"5a756caa-fee8-436f-ba39-fca2a13763fb\") " pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:27:56.016685 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:56.016654    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66zwh\" (UniqueName: \"kubernetes.io/projected/5a756caa-fee8-436f-ba39-fca2a13763fb-kube-api-access-66zwh\") pod \"maas-controller-769d7b9cbc-jsm54\" (UID: \"5a756caa-fee8-436f-ba39-fca2a13763fb\") " pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:27:56.081333 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:56.081275    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8548446449-chlpz"]
Jul 17 09:27:56.083566 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:27:56.083539    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c733f4_b43a_4a69_b443_306a214616da.slice/crio-8b9fa9189e81d3de2ec7f3241d18a3e5d81d7fba3194290091a165c0eef28ff6 WatchSource:0}: Error finding container 8b9fa9189e81d3de2ec7f3241d18a3e5d81d7fba3194290091a165c0eef28ff6: Status 404 returned error can't find the container with id 8b9fa9189e81d3de2ec7f3241d18a3e5d81d7fba3194290091a165c0eef28ff6
Jul 17 09:27:56.145031 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:56.144957    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:27:56.279631 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:56.279604    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-jsm54"]
Jul 17 09:27:56.280624 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:27:56.280601    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a756caa_fee8_436f_ba39_fca2a13763fb.slice/crio-8b1a7d3f525b1b20d85eaa8863bbc3d0b6e3b2ff3d5b6c5fd8485ee4a07918fd WatchSource:0}: Error finding container 8b1a7d3f525b1b20d85eaa8863bbc3d0b6e3b2ff3d5b6c5fd8485ee4a07918fd: Status 404 returned error can't find the container with id 8b1a7d3f525b1b20d85eaa8863bbc3d0b6e3b2ff3d5b6c5fd8485ee4a07918fd
Jul 17 09:27:56.447535 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:56.447500    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769d7b9cbc-jsm54" event={"ID":"5a756caa-fee8-436f-ba39-fca2a13763fb","Type":"ContainerStarted","Data":"8b1a7d3f525b1b20d85eaa8863bbc3d0b6e3b2ff3d5b6c5fd8485ee4a07918fd"}
Jul 17 09:27:56.448865 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:56.448839    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8548446449-chlpz" event={"ID":"15c733f4-b43a-4a69-b443-306a214616da","Type":"ContainerStarted","Data":"8b9fa9189e81d3de2ec7f3241d18a3e5d81d7fba3194290091a165c0eef28ff6"}
Jul 17 09:27:59.474538 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:59.474499    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769d7b9cbc-jsm54" event={"ID":"5a756caa-fee8-436f-ba39-fca2a13763fb","Type":"ContainerStarted","Data":"44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a"}
Jul 17 09:27:59.474976 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:59.474609    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:27:59.476154 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:59.476131    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8548446449-chlpz" event={"ID":"15c733f4-b43a-4a69-b443-306a214616da","Type":"ContainerStarted","Data":"b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4"}
Jul 17 09:27:59.476293 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:59.476176    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:27:59.513548 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:59.513497    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-769d7b9cbc-jsm54" podStartSLOduration=1.867365604 podStartE2EDuration="4.513483748s" totalImagesPullingTime="2.646118144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:59.512185033 +0000 UTC m=+563.577005755" watchObservedRunningTime="2026-07-17 09:27:59.513483748 +0000 UTC m=+563.578304447"
Jul 17 09:27:59.548068 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:27:59.548018    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8548446449-chlpz" podStartSLOduration=1.693809204 podStartE2EDuration="4.548002544s" totalImagesPullingTime="2.85419334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:59.545876906 +0000 UTC m=+563.610697643" watchObservedRunningTime="2026-07-17 09:27:59.548002544 +0000 UTC m=+563.612823266"
Jul 17 09:28:10.484504 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:10.484441    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:28:10.484913 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:10.484645    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:28:10.558604 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:10.558564    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8548446449-chlpz"]
Jul 17 09:28:10.558818 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:10.558793    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8548446449-chlpz" podUID="15c733f4-b43a-4a69-b443-306a214616da" containerName="manager" containerID="cri-o://b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4" gracePeriod=30
Jul 17 09:28:10.813073 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:10.813045    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:28:10.839730 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:10.839678    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmnwc\" (UniqueName: \"kubernetes.io/projected/15c733f4-b43a-4a69-b443-306a214616da-kube-api-access-wmnwc\") pod \"15c733f4-b43a-4a69-b443-306a214616da\" (UID: \"15c733f4-b43a-4a69-b443-306a214616da\") "
Jul 17 09:28:10.839879 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:10.839743    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15c733f4-b43a-4a69-b443-306a214616da-webhook-cert\") pod \"15c733f4-b43a-4a69-b443-306a214616da\" (UID: \"15c733f4-b43a-4a69-b443-306a214616da\") "
Jul 17 09:28:10.842451 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:10.842421    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c733f4-b43a-4a69-b443-306a214616da-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "15c733f4-b43a-4a69-b443-306a214616da" (UID: "15c733f4-b43a-4a69-b443-306a214616da"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:28:10.842598 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:10.842483    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15c733f4-b43a-4a69-b443-306a214616da-kube-api-access-wmnwc" (OuterVolumeSpecName: "kube-api-access-wmnwc") pod "15c733f4-b43a-4a69-b443-306a214616da" (UID: "15c733f4-b43a-4a69-b443-306a214616da"). InnerVolumeSpecName "kube-api-access-wmnwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:28:10.941408 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:10.941378    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/15c733f4-b43a-4a69-b443-306a214616da-webhook-cert\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:28:10.941408 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:10.941405    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmnwc\" (UniqueName: \"kubernetes.io/projected/15c733f4-b43a-4a69-b443-306a214616da-kube-api-access-wmnwc\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:28:11.543875 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:11.543838    2571 generic.go:356] "Generic (PLEG): container finished" podID="15c733f4-b43a-4a69-b443-306a214616da" containerID="b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4" exitCode=0
Jul 17 09:28:11.544258 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:11.543884    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8548446449-chlpz"
Jul 17 09:28:11.544258 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:11.543927    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8548446449-chlpz" event={"ID":"15c733f4-b43a-4a69-b443-306a214616da","Type":"ContainerDied","Data":"b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4"}
Jul 17 09:28:11.544258 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:11.543981    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8548446449-chlpz" event={"ID":"15c733f4-b43a-4a69-b443-306a214616da","Type":"ContainerDied","Data":"8b9fa9189e81d3de2ec7f3241d18a3e5d81d7fba3194290091a165c0eef28ff6"}
Jul 17 09:28:11.544258 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:11.543998    2571 scope.go:117] "RemoveContainer" containerID="b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4"
Jul 17 09:28:11.557958 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:11.557936    2571 scope.go:117] "RemoveContainer" containerID="b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4"
Jul 17 09:28:11.558238 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:28:11.558218    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4\": container with ID starting with b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4 not found: ID does not exist" containerID="b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4"
Jul 17 09:28:11.558278 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:11.558248    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4"} err="failed to get container status \"b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4\": rpc error: code = NotFound desc = could not find container \"b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4\": container with ID starting with b28017f5e4ea9e5439f7a051bd1dbfc96c0405da2c3e1429ba5cdb85d2ef99f4 not found: ID does not exist"
Jul 17 09:28:11.595194 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:11.595163    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8548446449-chlpz"]
Jul 17 09:28:11.605965 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:11.605940    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8548446449-chlpz"]
Jul 17 09:28:11.759736 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:11.759697    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15c733f4-b43a-4a69-b443-306a214616da" path="/var/lib/kubelet/pods/15c733f4-b43a-4a69-b443-306a214616da/volumes"
Jul 17 09:28:31.003766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.003725    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"]
Jul 17 09:28:31.004356 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.004326    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c733f4-b43a-4a69-b443-306a214616da" containerName="manager"
Jul 17 09:28:31.004356 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.004347    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c733f4-b43a-4a69-b443-306a214616da" containerName="manager"
Jul 17 09:28:31.004522 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.004493    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c733f4-b43a-4a69-b443-306a214616da" containerName="manager"
Jul 17 09:28:31.007455 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.007435    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:31.010506 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.010488    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 17 09:28:31.011538 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.011502    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m2p5w\""
Jul 17 09:28:31.020165 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.020146    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"]
Jul 17 09:28:31.117922 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.117886    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e727bc87-1596-4a13-b51a-72d3323e2886-openshift-service-ca\") pod \"maas-api-57ddf56f55-kj8cc\" (UID: \"e727bc87-1596-4a13-b51a-72d3323e2886\") " pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:31.118067 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.117956    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qm6v\" (UniqueName: \"kubernetes.io/projected/e727bc87-1596-4a13-b51a-72d3323e2886-kube-api-access-7qm6v\") pod \"maas-api-57ddf56f55-kj8cc\" (UID: \"e727bc87-1596-4a13-b51a-72d3323e2886\") " pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:31.118067 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.117979    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e727bc87-1596-4a13-b51a-72d3323e2886-maas-api-tls\") pod \"maas-api-57ddf56f55-kj8cc\" (UID: \"e727bc87-1596-4a13-b51a-72d3323e2886\") " pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:31.219080 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.219048    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qm6v\" (UniqueName: \"kubernetes.io/projected/e727bc87-1596-4a13-b51a-72d3323e2886-kube-api-access-7qm6v\") pod \"maas-api-57ddf56f55-kj8cc\" (UID: \"e727bc87-1596-4a13-b51a-72d3323e2886\") " pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:31.219080 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.219080    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e727bc87-1596-4a13-b51a-72d3323e2886-maas-api-tls\") pod \"maas-api-57ddf56f55-kj8cc\" (UID: \"e727bc87-1596-4a13-b51a-72d3323e2886\") " pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:31.219266 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.219136    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e727bc87-1596-4a13-b51a-72d3323e2886-openshift-service-ca\") pod \"maas-api-57ddf56f55-kj8cc\" (UID: \"e727bc87-1596-4a13-b51a-72d3323e2886\") " pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:31.219828 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.219803    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e727bc87-1596-4a13-b51a-72d3323e2886-openshift-service-ca\") pod \"maas-api-57ddf56f55-kj8cc\" (UID: \"e727bc87-1596-4a13-b51a-72d3323e2886\") " pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:31.221593 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.221566    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e727bc87-1596-4a13-b51a-72d3323e2886-maas-api-tls\") pod \"maas-api-57ddf56f55-kj8cc\" (UID: \"e727bc87-1596-4a13-b51a-72d3323e2886\") " pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:31.228667 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.228641    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qm6v\" (UniqueName: \"kubernetes.io/projected/e727bc87-1596-4a13-b51a-72d3323e2886-kube-api-access-7qm6v\") pod \"maas-api-57ddf56f55-kj8cc\" (UID: \"e727bc87-1596-4a13-b51a-72d3323e2886\") " pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:31.319921 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.319839    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:31.467939 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.467912    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"]
Jul 17 09:28:31.468446 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:28:31.468425    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode727bc87_1596_4a13_b51a_72d3323e2886.slice/crio-211437945fe81f14e6b1dba838da96d4c46e77b5b2dcb711d5604e86dafa016a WatchSource:0}: Error finding container 211437945fe81f14e6b1dba838da96d4c46e77b5b2dcb711d5604e86dafa016a: Status 404 returned error can't find the container with id 211437945fe81f14e6b1dba838da96d4c46e77b5b2dcb711d5604e86dafa016a
Jul 17 09:28:31.596274 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.596193    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-lh5rz"]
Jul 17 09:28:31.604075 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.602204    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz"
Jul 17 09:28:31.605700 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.605678    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 17 09:28:31.605968 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.605944    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rbq6p\""
Jul 17 09:28:31.618759 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.618732    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-lh5rz"]
Jul 17 09:28:31.658821 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.658788    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc" event={"ID":"e727bc87-1596-4a13-b51a-72d3323e2886","Type":"ContainerStarted","Data":"211437945fe81f14e6b1dba838da96d4c46e77b5b2dcb711d5604e86dafa016a"}
Jul 17 09:28:31.667862 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.667830    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xth9s"]
Jul 17 09:28:31.671603 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.671585    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s"
Jul 17 09:28:31.687258 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.687233    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xth9s"]
Jul 17 09:28:31.724324 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.724284    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ncrj\" (UniqueName: \"kubernetes.io/projected/505c6160-56a3-4353-973f-b460a979e97a-kube-api-access-7ncrj\") pod \"payload-pre-processing-7677b774bb-xth9s\" (UID: \"505c6160-56a3-4353-973f-b460a979e97a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s"
Jul 17 09:28:31.724509 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.724347    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/505c6160-56a3-4353-973f-b460a979e97a-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xth9s\" (UID: \"505c6160-56a3-4353-973f-b460a979e97a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s"
Jul 17 09:28:31.724509 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.724425    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ac5b6c-43fb-44fa-bbca-077b309bd8b8-plugins-config-volume\") pod \"payload-processing-5bf47cd988-lh5rz\" (UID: \"12ac5b6c-43fb-44fa-bbca-077b309bd8b8\") " pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz"
Jul 17 09:28:31.724664 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.724573    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27xgg\" (UniqueName: \"kubernetes.io/projected/12ac5b6c-43fb-44fa-bbca-077b309bd8b8-kube-api-access-27xgg\") pod \"payload-processing-5bf47cd988-lh5rz\" (UID: \"12ac5b6c-43fb-44fa-bbca-077b309bd8b8\") " pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz"
Jul 17 09:28:31.825169 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.825133    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ncrj\" (UniqueName: \"kubernetes.io/projected/505c6160-56a3-4353-973f-b460a979e97a-kube-api-access-7ncrj\") pod \"payload-pre-processing-7677b774bb-xth9s\" (UID: \"505c6160-56a3-4353-973f-b460a979e97a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s"
Jul 17 09:28:31.825169 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.825175    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/505c6160-56a3-4353-973f-b460a979e97a-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xth9s\" (UID: \"505c6160-56a3-4353-973f-b460a979e97a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s"
Jul 17 09:28:31.825447 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.825432    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ac5b6c-43fb-44fa-bbca-077b309bd8b8-plugins-config-volume\") pod \"payload-processing-5bf47cd988-lh5rz\" (UID: \"12ac5b6c-43fb-44fa-bbca-077b309bd8b8\") " pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz"
Jul 17 09:28:31.825557 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.825542    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27xgg\" (UniqueName: \"kubernetes.io/projected/12ac5b6c-43fb-44fa-bbca-077b309bd8b8-kube-api-access-27xgg\") pod \"payload-processing-5bf47cd988-lh5rz\" (UID: \"12ac5b6c-43fb-44fa-bbca-077b309bd8b8\") " pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz"
Jul 17 09:28:31.826082 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.826055    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/505c6160-56a3-4353-973f-b460a979e97a-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xth9s\" (UID: \"505c6160-56a3-4353-973f-b460a979e97a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s"
Jul 17 09:28:31.826210 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.826184    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12ac5b6c-43fb-44fa-bbca-077b309bd8b8-plugins-config-volume\") pod \"payload-processing-5bf47cd988-lh5rz\" (UID: \"12ac5b6c-43fb-44fa-bbca-077b309bd8b8\") " pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz"
Jul 17 09:28:31.835085 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.835061    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ncrj\" (UniqueName: \"kubernetes.io/projected/505c6160-56a3-4353-973f-b460a979e97a-kube-api-access-7ncrj\") pod \"payload-pre-processing-7677b774bb-xth9s\" (UID: \"505c6160-56a3-4353-973f-b460a979e97a\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s"
Jul 17 09:28:31.835177 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.835129    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27xgg\" (UniqueName: \"kubernetes.io/projected/12ac5b6c-43fb-44fa-bbca-077b309bd8b8-kube-api-access-27xgg\") pod \"payload-processing-5bf47cd988-lh5rz\" (UID: \"12ac5b6c-43fb-44fa-bbca-077b309bd8b8\") " pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz"
Jul 17 09:28:31.914724 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.914665    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz"
Jul 17 09:28:31.981828 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:31.981796    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s"
Jul 17 09:28:32.071039 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:32.070940    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-lh5rz"]
Jul 17 09:28:32.076134 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:28:32.076091    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ac5b6c_43fb_44fa_bbca_077b309bd8b8.slice/crio-aa953e6e8e4c547c453dd839007704184e34c4e63b20de91aec92f2a70f41fde WatchSource:0}: Error finding container aa953e6e8e4c547c453dd839007704184e34c4e63b20de91aec92f2a70f41fde: Status 404 returned error can't find the container with id aa953e6e8e4c547c453dd839007704184e34c4e63b20de91aec92f2a70f41fde
Jul 17 09:28:32.179941 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:32.179914    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xth9s"]
Jul 17 09:28:32.181910 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:28:32.181878    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod505c6160_56a3_4353_973f_b460a979e97a.slice/crio-88b34bab09252642efddb22e3cde9c2746ef9e9a8f2b35d5eff4ed6b861fe2f9 WatchSource:0}: Error finding container 88b34bab09252642efddb22e3cde9c2746ef9e9a8f2b35d5eff4ed6b861fe2f9: Status 404 returned error can't find the container with id 88b34bab09252642efddb22e3cde9c2746ef9e9a8f2b35d5eff4ed6b861fe2f9
Jul 17 09:28:32.667896 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:32.667845    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz" event={"ID":"12ac5b6c-43fb-44fa-bbca-077b309bd8b8","Type":"ContainerStarted","Data":"aa953e6e8e4c547c453dd839007704184e34c4e63b20de91aec92f2a70f41fde"}
Jul 17 09:28:32.669438 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:32.669382    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s" event={"ID":"505c6160-56a3-4353-973f-b460a979e97a","Type":"ContainerStarted","Data":"88b34bab09252642efddb22e3cde9c2746ef9e9a8f2b35d5eff4ed6b861fe2f9"}
Jul 17 09:28:33.679898 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:33.679828    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc" event={"ID":"e727bc87-1596-4a13-b51a-72d3323e2886","Type":"ContainerStarted","Data":"c8f569d2929cab4ae772218db33b844477215aa942f4c5590e78eabc32e687ff"}
Jul 17 09:28:33.680342 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:33.680026    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:33.720316 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:33.720254    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc" podStartSLOduration=2.197963476 podStartE2EDuration="3.720233452s" totalImagesPullingTime="1.522269976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:33.714728395 +0000 UTC m=+597.779549115" watchObservedRunningTime="2026-07-17 09:28:33.720233452 +0000 UTC m=+597.785054175"
Jul 17 09:28:34.686820 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:34.686783    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s" event={"ID":"505c6160-56a3-4353-973f-b460a979e97a","Type":"ContainerStarted","Data":"a9aaa1c1c16d9772a9f1bdcb4a61ab7cb8bee96a394d610fe487bcf652af231a"}
Jul 17 09:28:34.687264 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:34.686947    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s"
Jul 17 09:28:34.688481 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:34.688446    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz" event={"ID":"12ac5b6c-43fb-44fa-bbca-077b309bd8b8","Type":"ContainerStarted","Data":"1c33519b2d3614c44504890b546ce1e630f8be62ed83cf51bb90ae277648d70a"}
Jul 17 09:28:34.688603 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:34.688511    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz"
Jul 17 09:28:34.712170 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:34.712122    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s" podStartSLOduration=2.10764007 podStartE2EDuration="3.712108649s" totalImagesPullingTime="1.604468579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:34.710004593 +0000 UTC m=+598.774825314" watchObservedRunningTime="2026-07-17 09:28:34.712108649 +0000 UTC m=+598.776929416"
Jul 17 09:28:34.735636 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:34.735573    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz" podStartSLOduration=1.943562585 podStartE2EDuration="3.735552056s" totalImagesPullingTime="1.791989471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:34.732379181 +0000 UTC m=+598.797199903" watchObservedRunningTime="2026-07-17 09:28:34.735552056 +0000 UTC m=+598.800372780"
Jul 17 09:28:37.308104 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:37.308074    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:28:37.308544 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:37.308074    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:28:39.694495 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:39.694450    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-57ddf56f55-kj8cc"
Jul 17 09:28:45.697194 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:45.697160    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-lh5rz"
Jul 17 09:28:45.697576 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:28:45.697345    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-xth9s"
Jul 17 09:29:13.207275 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.207073    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"]
Jul 17 09:29:13.213870 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.213851    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.219830 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.219796    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mj6v8\""
Jul 17 09:29:13.220413 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.220395    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 09:29:13.220608 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.220590    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 09:29:13.225101 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.225080    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 17 09:29:13.255488 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.255421    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"]
Jul 17 09:29:13.303054 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.303021    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.303193 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.303074    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.303193 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.303098    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf5dcf60-6004-4056-96dd-d2f22070a73f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.303193 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.303122    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfvwh\" (UniqueName: \"kubernetes.io/projected/bf5dcf60-6004-4056-96dd-d2f22070a73f-kube-api-access-xfvwh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.303193 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.303144    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.303193 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.303163    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.404601 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.404563    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.404601 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.404604    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf5dcf60-6004-4056-96dd-d2f22070a73f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.404791 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.404628    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfvwh\" (UniqueName: \"kubernetes.io/projected/bf5dcf60-6004-4056-96dd-d2f22070a73f-kube-api-access-xfvwh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.404791 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.404664    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.404791 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.404693    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.404791 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.404777    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.405043 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.405022    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.405145 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.405038    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.405145 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.405083    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.406856 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.406832    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf5dcf60-6004-4056-96dd-d2f22070a73f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.407045 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.407028    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf5dcf60-6004-4056-96dd-d2f22070a73f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.415869 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.415843    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfvwh\" (UniqueName: \"kubernetes.io/projected/bf5dcf60-6004-4056-96dd-d2f22070a73f-kube-api-access-xfvwh\") pod \"e2e-distinct-simulated-kserve-59975d69f9-fw775\" (UID: \"bf5dcf60-6004-4056-96dd-d2f22070a73f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.525945 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.525869    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:13.685565 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.685538    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"]
Jul 17 09:29:13.687626 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:29:13.687592    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf5dcf60_6004_4056_96dd_d2f22070a73f.slice/crio-b164cdf84201d96a9d7f21537dc5be105ab03c0f222456052c23d57197ad467b WatchSource:0}: Error finding container b164cdf84201d96a9d7f21537dc5be105ab03c0f222456052c23d57197ad467b: Status 404 returned error can't find the container with id b164cdf84201d96a9d7f21537dc5be105ab03c0f222456052c23d57197ad467b
Jul 17 09:29:13.689028 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.689009    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:29:13.902349 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:13.902263    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775" event={"ID":"bf5dcf60-6004-4056-96dd-d2f22070a73f","Type":"ContainerStarted","Data":"b164cdf84201d96a9d7f21537dc5be105ab03c0f222456052c23d57197ad467b"}
Jul 17 09:29:14.909273 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:14.909240    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775" event={"ID":"bf5dcf60-6004-4056-96dd-d2f22070a73f","Type":"ContainerStarted","Data":"03383fcc5fa1e3d11cb2d12361207e7241bf28a05d4e2ccf2e559ecb6995dcc1"}
Jul 17 09:29:14.936073 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:14.936018    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775" podStartSLOduration=0.874759319 podStartE2EDuration="1.93599864s" totalImagesPullingTime="1.061239321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:29:14.932218495 +0000 UTC m=+638.997039221" watchObservedRunningTime="2026-07-17 09:29:14.93599864 +0000 UTC m=+639.000819366"
Jul 17 09:29:23.526481 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:23.526424    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:23.526850 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:23.526525    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:23.539340 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:23.539314    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:23.969847 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:23.969823    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-fw775"
Jul 17 09:29:35.304302 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.304260    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"]
Jul 17 09:29:35.308144 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.308123    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.310877 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.310858    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 17 09:29:35.319767 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.319742    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"]
Jul 17 09:29:35.400592 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.400554    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.400770 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.400611    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.400770 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.400639    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dac0dea-f7b5-47d6-a142-59c1182a313e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.400770 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.400682    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cf7v\" (UniqueName: \"kubernetes.io/projected/2dac0dea-f7b5-47d6-a142-59c1182a313e-kube-api-access-8cf7v\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.400770 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.400762    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.400931 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.400816    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.501994 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.501956    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.502184 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.502024    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.502184 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.502080    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.502184 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.502116    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.502184 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.502141    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dac0dea-f7b5-47d6-a142-59c1182a313e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.502370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.502190    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cf7v\" (UniqueName: \"kubernetes.io/projected/2dac0dea-f7b5-47d6-a142-59c1182a313e-kube-api-access-8cf7v\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.502518 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.502490    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.502601 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.502529    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.502601 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.502570    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.504389 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.504370    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dac0dea-f7b5-47d6-a142-59c1182a313e-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.504698 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.504676    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dac0dea-f7b5-47d6-a142-59c1182a313e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.524260 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.524229    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cf7v\" (UniqueName: \"kubernetes.io/projected/2dac0dea-f7b5-47d6-a142-59c1182a313e-kube-api-access-8cf7v\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bs66m\" (UID: \"2dac0dea-f7b5-47d6-a142-59c1182a313e\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.619454 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.619368    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:35.761646 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:35.761617    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"]
Jul 17 09:29:35.762660 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:29:35.762622    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dac0dea_f7b5_47d6_a142_59c1182a313e.slice/crio-a1fb798ef6670518a713d2dbc7ff5c51274ee66ff4eed1987a3d88f8d080fbfd WatchSource:0}: Error finding container a1fb798ef6670518a713d2dbc7ff5c51274ee66ff4eed1987a3d88f8d080fbfd: Status 404 returned error can't find the container with id a1fb798ef6670518a713d2dbc7ff5c51274ee66ff4eed1987a3d88f8d080fbfd
Jul 17 09:29:36.023899 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:36.023857    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m" event={"ID":"2dac0dea-f7b5-47d6-a142-59c1182a313e","Type":"ContainerStarted","Data":"a1fb798ef6670518a713d2dbc7ff5c51274ee66ff4eed1987a3d88f8d080fbfd"}
Jul 17 09:29:37.030797 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:37.030756    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m" event={"ID":"2dac0dea-f7b5-47d6-a142-59c1182a313e","Type":"ContainerStarted","Data":"4a4b762dbe95b5dd435490797601b5f2d828f1dcf74dba6512047c4afd843026"}
Jul 17 09:29:37.053108 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:37.053054    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m" podStartSLOduration=1.437197644 podStartE2EDuration="2.053027567s" totalImagesPullingTime="615.829923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:29:37.051965814 +0000 UTC m=+661.116786538" watchObservedRunningTime="2026-07-17 09:29:37.053027567 +0000 UTC m=+661.117848328"
Jul 17 09:29:45.619843 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:45.619804    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:45.620421 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:45.619890    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:45.632963 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:45.632937    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:29:46.090368 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:29:46.090337    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bs66m"
Jul 17 09:30:00.156480 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:00.156427    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r"]
Jul 17 09:30:00.160038 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:00.160018    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r"
Jul 17 09:30:00.162891 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:00.162868    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:30:00.169776 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:00.169759    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r"
Jul 17 09:30:00.174394 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:00.174368    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r"]
Jul 17 09:30:00.309806 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:00.309772    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r"]
Jul 17 09:30:00.311344 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:30:00.311320    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec724ac3_4239_4e4a_aa85_c063bd13c0c4.slice/crio-a5fe81f56f6c2b4bd9f44d6d0cb1a5821c1af71497d21d2ec489b75b7b8ac9ed WatchSource:0}: Error finding container a5fe81f56f6c2b4bd9f44d6d0cb1a5821c1af71497d21d2ec489b75b7b8ac9ed: Status 404 returned error can't find the container with id a5fe81f56f6c2b4bd9f44d6d0cb1a5821c1af71497d21d2ec489b75b7b8ac9ed
Jul 17 09:30:01.171069 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:01.171032    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r" event={"ID":"ec724ac3-4239-4e4a-aa85-c063bd13c0c4","Type":"ContainerStarted","Data":"a5fe81f56f6c2b4bd9f44d6d0cb1a5821c1af71497d21d2ec489b75b7b8ac9ed"}
Jul 17 09:30:03.184770 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:03.184695    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r" event={"ID":"ec724ac3-4239-4e4a-aa85-c063bd13c0c4","Type":"ContainerStarted","Data":"320450a7d1e6979bef5bd9ffd9e005a14c0b1831595973151293fc9a5f412dd3"}
Jul 17 09:30:03.203752 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:03.203706    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r" podStartSLOduration=0.582914045 podStartE2EDuration="3.203693319s" totalImagesPullingTime="2.620779274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:30:03.202825167 +0000 UTC m=+687.267645893" watchObservedRunningTime="2026-07-17 09:30:03.203693319 +0000 UTC m=+687.268514038"
Jul 17 09:30:04.192445 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:04.192414    2571 generic.go:356] "Generic (PLEG): container finished" podID="ec724ac3-4239-4e4a-aa85-c063bd13c0c4" containerID="320450a7d1e6979bef5bd9ffd9e005a14c0b1831595973151293fc9a5f412dd3" exitCode=0
Jul 17 09:30:04.192855 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:04.192475    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r" event={"ID":"ec724ac3-4239-4e4a-aa85-c063bd13c0c4","Type":"ContainerDied","Data":"320450a7d1e6979bef5bd9ffd9e005a14c0b1831595973151293fc9a5f412dd3"}
Jul 17 09:30:05.349105 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:05.349072    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r"
Jul 17 09:30:06.204780 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:06.204743    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r" event={"ID":"ec724ac3-4239-4e4a-aa85-c063bd13c0c4","Type":"ContainerDied","Data":"a5fe81f56f6c2b4bd9f44d6d0cb1a5821c1af71497d21d2ec489b75b7b8ac9ed"}
Jul 17 09:30:06.204780 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:06.204775    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r"
Jul 17 09:30:06.204978 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:30:06.204801    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5fe81f56f6c2b4bd9f44d6d0cb1a5821c1af71497d21d2ec489b75b7b8ac9ed"
Jul 17 09:32:05.926117 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:05.926021    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-jsm54"]
Jul 17 09:32:05.926923 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:05.926323    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-769d7b9cbc-jsm54" podUID="5a756caa-fee8-436f-ba39-fca2a13763fb" containerName="manager" containerID="cri-o://44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a" gracePeriod=30
Jul 17 09:32:06.187309 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.187218    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:32:06.255636 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.255602    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a756caa-fee8-436f-ba39-fca2a13763fb-webhook-cert\") pod \"5a756caa-fee8-436f-ba39-fca2a13763fb\" (UID: \"5a756caa-fee8-436f-ba39-fca2a13763fb\") "
Jul 17 09:32:06.255800 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.255721    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66zwh\" (UniqueName: \"kubernetes.io/projected/5a756caa-fee8-436f-ba39-fca2a13763fb-kube-api-access-66zwh\") pod \"5a756caa-fee8-436f-ba39-fca2a13763fb\" (UID: \"5a756caa-fee8-436f-ba39-fca2a13763fb\") "
Jul 17 09:32:06.257696 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.257656    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a756caa-fee8-436f-ba39-fca2a13763fb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5a756caa-fee8-436f-ba39-fca2a13763fb" (UID: "5a756caa-fee8-436f-ba39-fca2a13763fb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:32:06.257838 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.257808    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a756caa-fee8-436f-ba39-fca2a13763fb-kube-api-access-66zwh" (OuterVolumeSpecName: "kube-api-access-66zwh") pod "5a756caa-fee8-436f-ba39-fca2a13763fb" (UID: "5a756caa-fee8-436f-ba39-fca2a13763fb"). InnerVolumeSpecName "kube-api-access-66zwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:32:06.356636 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.356598    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a756caa-fee8-436f-ba39-fca2a13763fb-webhook-cert\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:32:06.356636 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.356631    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66zwh\" (UniqueName: \"kubernetes.io/projected/5a756caa-fee8-436f-ba39-fca2a13763fb-kube-api-access-66zwh\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:32:06.878079 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.878040    2571 generic.go:356] "Generic (PLEG): container finished" podID="5a756caa-fee8-436f-ba39-fca2a13763fb" containerID="44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a" exitCode=0
Jul 17 09:32:06.878252 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.878086    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769d7b9cbc-jsm54"
Jul 17 09:32:06.878252 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.878135    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769d7b9cbc-jsm54" event={"ID":"5a756caa-fee8-436f-ba39-fca2a13763fb","Type":"ContainerDied","Data":"44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a"}
Jul 17 09:32:06.878252 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.878199    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769d7b9cbc-jsm54" event={"ID":"5a756caa-fee8-436f-ba39-fca2a13763fb","Type":"ContainerDied","Data":"8b1a7d3f525b1b20d85eaa8863bbc3d0b6e3b2ff3d5b6c5fd8485ee4a07918fd"}
Jul 17 09:32:06.878252 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.878229    2571 scope.go:117] "RemoveContainer" containerID="44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a"
Jul 17 09:32:06.889085 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.889057    2571 scope.go:117] "RemoveContainer" containerID="44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a"
Jul 17 09:32:06.889372 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:32:06.889350    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a\": container with ID starting with 44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a not found: ID does not exist" containerID="44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a"
Jul 17 09:32:06.889478 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.889379    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a"} err="failed to get container status \"44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a\": rpc error: code = NotFound desc = could not find container \"44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a\": container with ID starting with 44677de0798f67cb1d99e27185070f14ad314c78128171c6a6e939b54b3a3c0a not found: ID does not exist"
Jul 17 09:32:06.904268 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.904237    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-jsm54"]
Jul 17 09:32:06.908658 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:06.908627    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-jsm54"]
Jul 17 09:32:07.758384 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:32:07.758353    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a756caa-fee8-436f-ba39-fca2a13763fb" path="/var/lib/kubelet/pods/5a756caa-fee8-436f-ba39-fca2a13763fb/volumes"
Jul 17 09:33:07.712741 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.712701    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-x5thw"]
Jul 17 09:33:07.713168 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.713136    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a756caa-fee8-436f-ba39-fca2a13763fb" containerName="manager"
Jul 17 09:33:07.713168 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.713147    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a756caa-fee8-436f-ba39-fca2a13763fb" containerName="manager"
Jul 17 09:33:07.713168 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.713164    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec724ac3-4239-4e4a-aa85-c063bd13c0c4" containerName="cleanup"
Jul 17 09:33:07.713168 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.713170    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec724ac3-4239-4e4a-aa85-c063bd13c0c4" containerName="cleanup"
Jul 17 09:33:07.713299 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.713240    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec724ac3-4239-4e4a-aa85-c063bd13c0c4" containerName="cleanup"
Jul 17 09:33:07.713299 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.713251    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a756caa-fee8-436f-ba39-fca2a13763fb" containerName="manager"
Jul 17 09:33:07.716342 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.716326    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:07.719439 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.719410    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 09:33:07.720524 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.720502    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 09:33:07.720653 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.720529    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-52cp9\""
Jul 17 09:33:07.727915 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.727893    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-x5thw"]
Jul 17 09:33:07.819788 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.819755    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecf0d886-df62-43d8-b067-a145431670a0-webhook-cert\") pod \"maas-controller-769d7b9cbc-x5thw\" (UID: \"ecf0d886-df62-43d8-b067-a145431670a0\") " pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:07.819788 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.819792    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vz2k\" (UniqueName: \"kubernetes.io/projected/ecf0d886-df62-43d8-b067-a145431670a0-kube-api-access-2vz2k\") pod \"maas-controller-769d7b9cbc-x5thw\" (UID: \"ecf0d886-df62-43d8-b067-a145431670a0\") " pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:07.920699 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.920659    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecf0d886-df62-43d8-b067-a145431670a0-webhook-cert\") pod \"maas-controller-769d7b9cbc-x5thw\" (UID: \"ecf0d886-df62-43d8-b067-a145431670a0\") " pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:07.920699 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.920701    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vz2k\" (UniqueName: \"kubernetes.io/projected/ecf0d886-df62-43d8-b067-a145431670a0-kube-api-access-2vz2k\") pod \"maas-controller-769d7b9cbc-x5thw\" (UID: \"ecf0d886-df62-43d8-b067-a145431670a0\") " pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:07.923146 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.923115    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecf0d886-df62-43d8-b067-a145431670a0-webhook-cert\") pod \"maas-controller-769d7b9cbc-x5thw\" (UID: \"ecf0d886-df62-43d8-b067-a145431670a0\") " pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:07.930718 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:07.930694    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vz2k\" (UniqueName: \"kubernetes.io/projected/ecf0d886-df62-43d8-b067-a145431670a0-kube-api-access-2vz2k\") pod \"maas-controller-769d7b9cbc-x5thw\" (UID: \"ecf0d886-df62-43d8-b067-a145431670a0\") " pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:08.028109 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:08.028014    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:08.169841 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:08.169810    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-x5thw"]
Jul 17 09:33:08.170924 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:33:08.170883    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf0d886_df62_43d8_b067_a145431670a0.slice/crio-a550aa46511eeab09cedf2681bbc8f45b90938b9e7af27ace98972a71c37c05f WatchSource:0}: Error finding container a550aa46511eeab09cedf2681bbc8f45b90938b9e7af27ace98972a71c37c05f: Status 404 returned error can't find the container with id a550aa46511eeab09cedf2681bbc8f45b90938b9e7af27ace98972a71c37c05f
Jul 17 09:33:08.225780 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:08.225735    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769d7b9cbc-x5thw" event={"ID":"ecf0d886-df62-43d8-b067-a145431670a0","Type":"ContainerStarted","Data":"a550aa46511eeab09cedf2681bbc8f45b90938b9e7af27ace98972a71c37c05f"}
Jul 17 09:33:09.232732 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:09.232691    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769d7b9cbc-x5thw" event={"ID":"ecf0d886-df62-43d8-b067-a145431670a0","Type":"ContainerStarted","Data":"0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf"}
Jul 17 09:33:09.233196 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:09.232817    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:09.254772 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:09.254720    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-769d7b9cbc-x5thw" podStartSLOduration=1.829121341 podStartE2EDuration="2.254705286s" totalImagesPullingTime="425.583945ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:33:09.253269391 +0000 UTC m=+873.318090113" watchObservedRunningTime="2026-07-17 09:33:09.254705286 +0000 UTC m=+873.319526036"
Jul 17 09:33:20.240841 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:20.240806    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:24.883894 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:24.883841    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-x5thw"]
Jul 17 09:33:24.884282 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:24.884067    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-769d7b9cbc-x5thw" podUID="ecf0d886-df62-43d8-b067-a145431670a0" containerName="manager" containerID="cri-o://0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf" gracePeriod=30
Jul 17 09:33:25.167085 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.167060    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:25.174881 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.174857    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vz2k\" (UniqueName: \"kubernetes.io/projected/ecf0d886-df62-43d8-b067-a145431670a0-kube-api-access-2vz2k\") pod \"ecf0d886-df62-43d8-b067-a145431670a0\" (UID: \"ecf0d886-df62-43d8-b067-a145431670a0\") "
Jul 17 09:33:25.174998 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.174913    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecf0d886-df62-43d8-b067-a145431670a0-webhook-cert\") pod \"ecf0d886-df62-43d8-b067-a145431670a0\" (UID: \"ecf0d886-df62-43d8-b067-a145431670a0\") "
Jul 17 09:33:25.176848 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.176819    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecf0d886-df62-43d8-b067-a145431670a0-kube-api-access-2vz2k" (OuterVolumeSpecName: "kube-api-access-2vz2k") pod "ecf0d886-df62-43d8-b067-a145431670a0" (UID: "ecf0d886-df62-43d8-b067-a145431670a0"). InnerVolumeSpecName "kube-api-access-2vz2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:33:25.176848 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.176824    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecf0d886-df62-43d8-b067-a145431670a0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ecf0d886-df62-43d8-b067-a145431670a0" (UID: "ecf0d886-df62-43d8-b067-a145431670a0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:33:25.276111 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.276081    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vz2k\" (UniqueName: \"kubernetes.io/projected/ecf0d886-df62-43d8-b067-a145431670a0-kube-api-access-2vz2k\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:33:25.276111 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.276108    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecf0d886-df62-43d8-b067-a145431670a0-webhook-cert\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:33:25.322284 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.322254    2571 generic.go:356] "Generic (PLEG): container finished" podID="ecf0d886-df62-43d8-b067-a145431670a0" containerID="0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf" exitCode=0
Jul 17 09:33:25.322435 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.322307    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769d7b9cbc-x5thw"
Jul 17 09:33:25.322435 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.322332    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769d7b9cbc-x5thw" event={"ID":"ecf0d886-df62-43d8-b067-a145431670a0","Type":"ContainerDied","Data":"0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf"}
Jul 17 09:33:25.322435 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.322415    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769d7b9cbc-x5thw" event={"ID":"ecf0d886-df62-43d8-b067-a145431670a0","Type":"ContainerDied","Data":"a550aa46511eeab09cedf2681bbc8f45b90938b9e7af27ace98972a71c37c05f"}
Jul 17 09:33:25.322585 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.322439    2571 scope.go:117] "RemoveContainer" containerID="0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf"
Jul 17 09:33:25.332883 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.332866    2571 scope.go:117] "RemoveContainer" containerID="0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf"
Jul 17 09:33:25.333139 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:33:25.333121    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf\": container with ID starting with 0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf not found: ID does not exist" containerID="0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf"
Jul 17 09:33:25.333195 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.333147    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf"} err="failed to get container status \"0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf\": rpc error: code = NotFound desc = could not find container \"0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf\": container with ID starting with 0182a9b7937945bbb3cdb2860d1d6764e2e59bdcabd4d506022928e7be7064bf not found: ID does not exist"
Jul 17 09:33:25.346457 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.346431    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-x5thw"]
Jul 17 09:33:25.350471 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.350440    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-x5thw"]
Jul 17 09:33:25.759912 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:25.759874    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecf0d886-df62-43d8-b067-a145431670a0" path="/var/lib/kubelet/pods/ecf0d886-df62-43d8-b067-a145431670a0/volumes"
Jul 17 09:33:26.146392 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.146315    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-6nlfc"]
Jul 17 09:33:26.146786 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.146772    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecf0d886-df62-43d8-b067-a145431670a0" containerName="manager"
Jul 17 09:33:26.146786 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.146788    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecf0d886-df62-43d8-b067-a145431670a0" containerName="manager"
Jul 17 09:33:26.146875 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.146866    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecf0d886-df62-43d8-b067-a145431670a0" containerName="manager"
Jul 17 09:33:26.151229 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.151211    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769d7b9cbc-6nlfc"
Jul 17 09:33:26.154875 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.154848    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 09:33:26.154995 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.154886    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 09:33:26.154995 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.154923    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-52cp9\""
Jul 17 09:33:26.167672 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.167647    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-6nlfc"]
Jul 17 09:33:26.180618 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.180593    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzd5g\" (UniqueName: \"kubernetes.io/projected/029c86fc-87ae-42fb-b508-9c967032c8cb-kube-api-access-jzd5g\") pod \"maas-controller-769d7b9cbc-6nlfc\" (UID: \"029c86fc-87ae-42fb-b508-9c967032c8cb\") " pod="opendatahub/maas-controller-769d7b9cbc-6nlfc"
Jul 17 09:33:26.180760 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.180629    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/029c86fc-87ae-42fb-b508-9c967032c8cb-webhook-cert\") pod \"maas-controller-769d7b9cbc-6nlfc\" (UID: \"029c86fc-87ae-42fb-b508-9c967032c8cb\") " pod="opendatahub/maas-controller-769d7b9cbc-6nlfc"
Jul 17 09:33:26.281389 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.281350    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzd5g\" (UniqueName: \"kubernetes.io/projected/029c86fc-87ae-42fb-b508-9c967032c8cb-kube-api-access-jzd5g\") pod \"maas-controller-769d7b9cbc-6nlfc\" (UID: \"029c86fc-87ae-42fb-b508-9c967032c8cb\") " pod="opendatahub/maas-controller-769d7b9cbc-6nlfc"
Jul 17 09:33:26.281609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.281398    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/029c86fc-87ae-42fb-b508-9c967032c8cb-webhook-cert\") pod \"maas-controller-769d7b9cbc-6nlfc\" (UID: \"029c86fc-87ae-42fb-b508-9c967032c8cb\") " pod="opendatahub/maas-controller-769d7b9cbc-6nlfc"
Jul 17 09:33:26.284060 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.284027    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/029c86fc-87ae-42fb-b508-9c967032c8cb-webhook-cert\") pod \"maas-controller-769d7b9cbc-6nlfc\" (UID: \"029c86fc-87ae-42fb-b508-9c967032c8cb\") " pod="opendatahub/maas-controller-769d7b9cbc-6nlfc"
Jul 17 09:33:26.291367 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.291341    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzd5g\" (UniqueName: \"kubernetes.io/projected/029c86fc-87ae-42fb-b508-9c967032c8cb-kube-api-access-jzd5g\") pod \"maas-controller-769d7b9cbc-6nlfc\" (UID: \"029c86fc-87ae-42fb-b508-9c967032c8cb\") " pod="opendatahub/maas-controller-769d7b9cbc-6nlfc"
Jul 17 09:33:26.461382 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.461354    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-769d7b9cbc-6nlfc"
Jul 17 09:33:26.605898 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:26.605874    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-769d7b9cbc-6nlfc"]
Jul 17 09:33:26.607458 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:33:26.607429    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029c86fc_87ae_42fb_b508_9c967032c8cb.slice/crio-3ce4b17c10ed368e4e03ed86f87128decbdb778119f17d88e4d558ff7f37e580 WatchSource:0}: Error finding container 3ce4b17c10ed368e4e03ed86f87128decbdb778119f17d88e4d558ff7f37e580: Status 404 returned error can't find the container with id 3ce4b17c10ed368e4e03ed86f87128decbdb778119f17d88e4d558ff7f37e580
Jul 17 09:33:27.340424 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:27.340312    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769d7b9cbc-6nlfc" event={"ID":"029c86fc-87ae-42fb-b508-9c967032c8cb","Type":"ContainerStarted","Data":"cb11b01b088e1a26064553c4402129edaa86f9d09d067f84582570713bf567ab"}
Jul 17 09:33:27.340424 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:27.340394    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-769d7b9cbc-6nlfc"
Jul 17 09:33:27.340424 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:27.340418    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-769d7b9cbc-6nlfc" event={"ID":"029c86fc-87ae-42fb-b508-9c967032c8cb","Type":"ContainerStarted","Data":"3ce4b17c10ed368e4e03ed86f87128decbdb778119f17d88e4d558ff7f37e580"}
Jul 17 09:33:27.362459 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:27.362409    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-769d7b9cbc-6nlfc" podStartSLOduration=0.958283153 podStartE2EDuration="1.362392649s" totalImagesPullingTime="404.109496ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:33:27.359883529 +0000 UTC m=+891.424704251" watchObservedRunningTime="2026-07-17 09:33:27.362392649 +0000 UTC m=+891.427213372"
Jul 17 09:33:37.348081 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:37.347996    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:33:37.348937 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:37.348919    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:33:38.348128 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:33:38.348094    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-769d7b9cbc-6nlfc"
Jul 17 09:38:37.386582 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:38:37.386540    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:38:37.389267 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:38:37.389246    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:43:37.423740 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:43:37.423708    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:43:37.428054 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:43:37.428033    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:45:49.524982 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.524939    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"]
Jul 17 09:45:49.528891 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.528872    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:49.533010 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.532988    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f15c7b6a-serving-cert\""
Jul 17 09:45:49.542796 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.542770    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"]
Jul 17 09:45:49.585814 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.585785    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e4575b8-3d97-44e4-b609-e76f012e90fe-openshift-service-ca\") pod \"maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:49.585930 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.585866    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e4575b8-3d97-44e4-b609-e76f012e90fe-maas-api-tls\") pod \"maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:49.585930 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.585893    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp6p5\" (UniqueName: \"kubernetes.io/projected/6e4575b8-3d97-44e4-b609-e76f012e90fe-kube-api-access-kp6p5\") pod \"maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:49.688489 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.687437    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e4575b8-3d97-44e4-b609-e76f012e90fe-maas-api-tls\") pod \"maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:49.688489 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.687509    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp6p5\" (UniqueName: \"kubernetes.io/projected/6e4575b8-3d97-44e4-b609-e76f012e90fe-kube-api-access-kp6p5\") pod \"maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:49.688489 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.687617    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e4575b8-3d97-44e4-b609-e76f012e90fe-openshift-service-ca\") pod \"maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:49.688489 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.688408    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e4575b8-3d97-44e4-b609-e76f012e90fe-openshift-service-ca\") pod \"maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:49.688832 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:45:49.688678    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-serving-cert: secret "maas-api-e2e-ait-f15c7b6a-serving-cert" not found
Jul 17 09:45:49.688832 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:45:49.688792    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e4575b8-3d97-44e4-b609-e76f012e90fe-maas-api-tls podName:6e4575b8-3d97-44e4-b609-e76f012e90fe nodeName:}" failed. No retries permitted until 2026-07-17 09:45:50.188772616 +0000 UTC m=+1634.253593330 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6e4575b8-3d97-44e4-b609-e76f012e90fe-maas-api-tls") pod "maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" (UID: "6e4575b8-3d97-44e4-b609-e76f012e90fe") : secret "maas-api-e2e-ait-f15c7b6a-serving-cert" not found
Jul 17 09:45:49.701380 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:49.701350    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp6p5\" (UniqueName: \"kubernetes.io/projected/6e4575b8-3d97-44e4-b609-e76f012e90fe-kube-api-access-kp6p5\") pod \"maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:50.192593 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:50.192557    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e4575b8-3d97-44e4-b609-e76f012e90fe-maas-api-tls\") pod \"maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:50.194893 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:50.194871    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e4575b8-3d97-44e4-b609-e76f012e90fe-maas-api-tls\") pod \"maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:50.442938 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:50.442865    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:50.599868 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:50.599838    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"]
Jul 17 09:45:50.601151 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:45:50.601121    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4575b8_3d97_44e4_b609_e76f012e90fe.slice/crio-dcd05a66dd2c92a76286df7761708830db47382d5cb8b58e43989b7dd9ae9f5e WatchSource:0}: Error finding container dcd05a66dd2c92a76286df7761708830db47382d5cb8b58e43989b7dd9ae9f5e: Status 404 returned error can't find the container with id dcd05a66dd2c92a76286df7761708830db47382d5cb8b58e43989b7dd9ae9f5e
Jul 17 09:45:50.602486 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:50.602455    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:45:51.522632 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:51.522598    2571 generic.go:356] "Generic (PLEG): container finished" podID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerID="258ac0a8bd7340c18f0abffdac5f12493adb1107c8b3b7b0e501ff4d3d3a7323" exitCode=1
Jul 17 09:45:51.522774 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:51.522653    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" event={"ID":"6e4575b8-3d97-44e4-b609-e76f012e90fe","Type":"ContainerDied","Data":"258ac0a8bd7340c18f0abffdac5f12493adb1107c8b3b7b0e501ff4d3d3a7323"}
Jul 17 09:45:51.522774 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:51.522744    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" event={"ID":"6e4575b8-3d97-44e4-b609-e76f012e90fe","Type":"ContainerStarted","Data":"dcd05a66dd2c92a76286df7761708830db47382d5cb8b58e43989b7dd9ae9f5e"}
Jul 17 09:45:51.522983 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:51.522970    2571 scope.go:117] "RemoveContainer" containerID="258ac0a8bd7340c18f0abffdac5f12493adb1107c8b3b7b0e501ff4d3d3a7323"
Jul 17 09:45:52.530145 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:52.530062    2571 generic.go:356] "Generic (PLEG): container finished" podID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerID="c3ad93acd92a4e6d61f81a1d5a7a85db861f13a19a1a460e8f2ee2154eecba80" exitCode=1
Jul 17 09:45:52.530604 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:52.530150    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" event={"ID":"6e4575b8-3d97-44e4-b609-e76f012e90fe","Type":"ContainerDied","Data":"c3ad93acd92a4e6d61f81a1d5a7a85db861f13a19a1a460e8f2ee2154eecba80"}
Jul 17 09:45:52.530604 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:52.530217    2571 scope.go:117] "RemoveContainer" containerID="258ac0a8bd7340c18f0abffdac5f12493adb1107c8b3b7b0e501ff4d3d3a7323"
Jul 17 09:45:52.530604 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:52.530386    2571 scope.go:117] "RemoveContainer" containerID="c3ad93acd92a4e6d61f81a1d5a7a85db861f13a19a1a460e8f2ee2154eecba80"
Jul 17 09:45:52.530735 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:45:52.530616    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:45:53.539181 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:53.539139    2571 scope.go:117] "RemoveContainer" containerID="c3ad93acd92a4e6d61f81a1d5a7a85db861f13a19a1a460e8f2ee2154eecba80"
Jul 17 09:45:53.539807 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:45:53.539324    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:45:55.443687 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:55.443655    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:45:55.444068 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:45:55.444036    2571 scope.go:117] "RemoveContainer" containerID="c3ad93acd92a4e6d61f81a1d5a7a85db861f13a19a1a460e8f2ee2154eecba80"
Jul 17 09:45:55.444229 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:45:55.444209    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:46:00.443609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:00.443580    2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:46:00.443983 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:00.443947    2571 scope.go:117] "RemoveContainer" containerID="c3ad93acd92a4e6d61f81a1d5a7a85db861f13a19a1a460e8f2ee2154eecba80"
Jul 17 09:46:00.444143 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:46:00.444123    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:46:11.752912 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:11.752881    2571 scope.go:117] "RemoveContainer" containerID="c3ad93acd92a4e6d61f81a1d5a7a85db861f13a19a1a460e8f2ee2154eecba80"
Jul 17 09:46:12.644703 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:12.644617    2571 generic.go:356] "Generic (PLEG): container finished" podID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerID="e39865d67510577487ebc4bf0dac326c9cfb0f7d72963ee617aaba140cdf8ff5" exitCode=1
Jul 17 09:46:12.644703 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:12.644676    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" event={"ID":"6e4575b8-3d97-44e4-b609-e76f012e90fe","Type":"ContainerDied","Data":"e39865d67510577487ebc4bf0dac326c9cfb0f7d72963ee617aaba140cdf8ff5"}
Jul 17 09:46:12.644893 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:12.644726    2571 scope.go:117] "RemoveContainer" containerID="c3ad93acd92a4e6d61f81a1d5a7a85db861f13a19a1a460e8f2ee2154eecba80"
Jul 17 09:46:12.645108 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:12.645088    2571 scope.go:117] "RemoveContainer" containerID="e39865d67510577487ebc4bf0dac326c9cfb0f7d72963ee617aaba140cdf8ff5"
Jul 17 09:46:12.645338 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:46:12.645318    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:46:15.443494 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:15.443444    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:46:15.443953 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:15.443891    2571 scope.go:117] "RemoveContainer" containerID="e39865d67510577487ebc4bf0dac326c9cfb0f7d72963ee617aaba140cdf8ff5"
Jul 17 09:46:15.444108 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:46:15.444084    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:46:20.443505 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:20.443456    2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:46:20.444113 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:20.443983    2571 scope.go:117] "RemoveContainer" containerID="e39865d67510577487ebc4bf0dac326c9cfb0f7d72963ee617aaba140cdf8ff5"
Jul 17 09:46:20.444254 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:46:20.444226    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:46:32.752686 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:32.752647    2571 scope.go:117] "RemoveContainer" containerID="e39865d67510577487ebc4bf0dac326c9cfb0f7d72963ee617aaba140cdf8ff5"
Jul 17 09:46:33.771565 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:33.771452    2571 generic.go:356] "Generic (PLEG): container finished" podID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerID="3fa2abb2c44e6f9f9cb952513055a61248a6b0d4d1e8489dfae5704fc9dd1c84" exitCode=1
Jul 17 09:46:33.771565 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:33.771502    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" event={"ID":"6e4575b8-3d97-44e4-b609-e76f012e90fe","Type":"ContainerDied","Data":"3fa2abb2c44e6f9f9cb952513055a61248a6b0d4d1e8489dfae5704fc9dd1c84"}
Jul 17 09:46:33.772077 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:33.771575    2571 scope.go:117] "RemoveContainer" containerID="e39865d67510577487ebc4bf0dac326c9cfb0f7d72963ee617aaba140cdf8ff5"
Jul 17 09:46:33.772077 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:33.771984    2571 scope.go:117] "RemoveContainer" containerID="3fa2abb2c44e6f9f9cb952513055a61248a6b0d4d1e8489dfae5704fc9dd1c84"
Jul 17 09:46:33.772231 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:46:33.772207    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:46:35.443354 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:35.443320    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:46:35.443771 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:35.443720    2571 scope.go:117] "RemoveContainer" containerID="3fa2abb2c44e6f9f9cb952513055a61248a6b0d4d1e8489dfae5704fc9dd1c84"
Jul 17 09:46:35.443978 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:46:35.443953    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:46:40.443446 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:40.443414    2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:46:40.443862 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:40.443793    2571 scope.go:117] "RemoveContainer" containerID="3fa2abb2c44e6f9f9cb952513055a61248a6b0d4d1e8489dfae5704fc9dd1c84"
Jul 17 09:46:40.444029 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:46:40.444006    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:46:55.753092 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:46:55.753059    2571 scope.go:117] "RemoveContainer" containerID="3fa2abb2c44e6f9f9cb952513055a61248a6b0d4d1e8489dfae5704fc9dd1c84"
Jul 17 09:46:55.753560 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:46:55.753273    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:47:06.752352 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:06.752269    2571 scope.go:117] "RemoveContainer" containerID="3fa2abb2c44e6f9f9cb952513055a61248a6b0d4d1e8489dfae5704fc9dd1c84"
Jul 17 09:47:06.752852 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:47:06.752497    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:47:21.752796 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:21.752765    2571 scope.go:117] "RemoveContainer" containerID="3fa2abb2c44e6f9f9cb952513055a61248a6b0d4d1e8489dfae5704fc9dd1c84"
Jul 17 09:47:23.052002 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:23.051969    2571 generic.go:356] "Generic (PLEG): container finished" podID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerID="fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a" exitCode=1
Jul 17 09:47:23.052418 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:23.052013    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" event={"ID":"6e4575b8-3d97-44e4-b609-e76f012e90fe","Type":"ContainerDied","Data":"fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a"}
Jul 17 09:47:23.052418 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:23.052060    2571 scope.go:117] "RemoveContainer" containerID="3fa2abb2c44e6f9f9cb952513055a61248a6b0d4d1e8489dfae5704fc9dd1c84"
Jul 17 09:47:23.052418 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:23.052379    2571 scope.go:117] "RemoveContainer" containerID="fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a"
Jul 17 09:47:23.052652 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:47:23.052631    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:47:25.443817 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:25.443781    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:47:25.444232 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:25.444212    2571 scope.go:117] "RemoveContainer" containerID="fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a"
Jul 17 09:47:25.444416 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:47:25.444395    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:47:30.443322 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:30.443286    2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:47:30.443769 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:30.443718    2571 scope.go:117] "RemoveContainer" containerID="fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a"
Jul 17 09:47:30.443948 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:47:30.443927    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:47:43.752741 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:43.752711    2571 scope.go:117] "RemoveContainer" containerID="fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a"
Jul 17 09:47:43.753210 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:47:43.752886    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:47:54.752853 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:47:54.752820    2571 scope.go:117] "RemoveContainer" containerID="fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a"
Jul 17 09:47:54.753233 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:47:54.753039    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:48:06.752965 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:06.752928    2571 scope.go:117] "RemoveContainer" containerID="fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a"
Jul 17 09:48:06.753415 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:48:06.753127    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:48:19.752211 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:19.752173    2571 scope.go:117] "RemoveContainer" containerID="fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a"
Jul 17 09:48:19.752832 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:48:19.752337    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:48:30.752655 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:30.752583    2571 scope.go:117] "RemoveContainer" containerID="fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a"
Jul 17 09:48:30.753111 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:48:30.752761    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:48:37.462622 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:37.462589    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:48:37.468875 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:37.468854    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:48:44.753201 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:44.753171    2571 scope.go:117] "RemoveContainer" containerID="fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a"
Jul 17 09:48:45.507435 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:45.507397    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" event={"ID":"6e4575b8-3d97-44e4-b609-e76f012e90fe","Type":"ContainerStarted","Data":"ac4dff3ba8367ade7696863d946e476fa9ed8c659bfd18bffdfa1ba9ae522672"}
Jul 17 09:48:45.507650 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:45.507633    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:48:45.534860 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:45.534806    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podStartSLOduration=173.887458583 podStartE2EDuration="2m56.534790677s" totalImagesPullingTime="2.647332094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:45:49 +0000 UTC" imagePullSessionsCount=6 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:48:45.531493075 +0000 UTC m=+1809.596313800" watchObservedRunningTime="2026-07-17 09:48:45.534790677 +0000 UTC m=+1809.599611399"
Jul 17 09:48:46.515398 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:46.515360    2571 generic.go:356] "Generic (PLEG): container finished" podID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerID="ac4dff3ba8367ade7696863d946e476fa9ed8c659bfd18bffdfa1ba9ae522672" exitCode=1
Jul 17 09:48:46.515849 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:46.515426    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" event={"ID":"6e4575b8-3d97-44e4-b609-e76f012e90fe","Type":"ContainerDied","Data":"ac4dff3ba8367ade7696863d946e476fa9ed8c659bfd18bffdfa1ba9ae522672"}
Jul 17 09:48:46.515849 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:46.515539    2571 scope.go:117] "RemoveContainer" containerID="fc0979f233802247b595fbdef7b59f27d3e85e7e243326670253cc97578f453a"
Jul 17 09:48:46.515849 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:46.515752    2571 scope.go:117] "RemoveContainer" containerID="ac4dff3ba8367ade7696863d946e476fa9ed8c659bfd18bffdfa1ba9ae522672"
Jul 17 09:48:46.516020 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:48:46.515999    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:48:47.523054 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:47.523020    2571 scope.go:117] "RemoveContainer" containerID="ac4dff3ba8367ade7696863d946e476fa9ed8c659bfd18bffdfa1ba9ae522672"
Jul 17 09:48:47.523433 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:48:47.523218    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:48:50.443810 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:50.443772    2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:48:50.444208 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:48:50.444194    2571 scope.go:117] "RemoveContainer" containerID="ac4dff3ba8367ade7696863d946e476fa9ed8c659bfd18bffdfa1ba9ae522672"
Jul 17 09:48:50.444386 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:48:50.444367    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=maas-api pod=maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd_odh-ai-gateway-infra(6e4575b8-3d97-44e4-b609-e76f012e90fe)\"" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe"
Jul 17 09:49:01.163212 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.163177    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"]
Jul 17 09:49:01.312243 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.312218    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:49:01.427092 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.426987    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp6p5\" (UniqueName: \"kubernetes.io/projected/6e4575b8-3d97-44e4-b609-e76f012e90fe-kube-api-access-kp6p5\") pod \"6e4575b8-3d97-44e4-b609-e76f012e90fe\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") "
Jul 17 09:49:01.427092 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.427049    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e4575b8-3d97-44e4-b609-e76f012e90fe-maas-api-tls\") pod \"6e4575b8-3d97-44e4-b609-e76f012e90fe\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") "
Jul 17 09:49:01.427092 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.427090    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e4575b8-3d97-44e4-b609-e76f012e90fe-openshift-service-ca\") pod \"6e4575b8-3d97-44e4-b609-e76f012e90fe\" (UID: \"6e4575b8-3d97-44e4-b609-e76f012e90fe\") "
Jul 17 09:49:01.427550 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.427526    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e4575b8-3d97-44e4-b609-e76f012e90fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6e4575b8-3d97-44e4-b609-e76f012e90fe" (UID: "6e4575b8-3d97-44e4-b609-e76f012e90fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:49:01.429248 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.429226    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e4575b8-3d97-44e4-b609-e76f012e90fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6e4575b8-3d97-44e4-b609-e76f012e90fe" (UID: "6e4575b8-3d97-44e4-b609-e76f012e90fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:49:01.429340 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.429327    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e4575b8-3d97-44e4-b609-e76f012e90fe-kube-api-access-kp6p5" (OuterVolumeSpecName: "kube-api-access-kp6p5") pod "6e4575b8-3d97-44e4-b609-e76f012e90fe" (UID: "6e4575b8-3d97-44e4-b609-e76f012e90fe"). InnerVolumeSpecName "kube-api-access-kp6p5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:49:01.528187 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.528136    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp6p5\" (UniqueName: \"kubernetes.io/projected/6e4575b8-3d97-44e4-b609-e76f012e90fe-kube-api-access-kp6p5\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:49:01.528187 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.528183    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e4575b8-3d97-44e4-b609-e76f012e90fe-maas-api-tls\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:49:01.528187 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.528194    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e4575b8-3d97-44e4-b609-e76f012e90fe-openshift-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:49:01.602186 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.602145    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd" event={"ID":"6e4575b8-3d97-44e4-b609-e76f012e90fe","Type":"ContainerDied","Data":"dcd05a66dd2c92a76286df7761708830db47382d5cb8b58e43989b7dd9ae9f5e"}
Jul 17 09:49:01.602186 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.602163    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"
Jul 17 09:49:01.602389 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.602208    2571 scope.go:117] "RemoveContainer" containerID="ac4dff3ba8367ade7696863d946e476fa9ed8c659bfd18bffdfa1ba9ae522672"
Jul 17 09:49:01.627354 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.627330    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"]
Jul 17 09:49:01.636849 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.636825    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f15c7b6a-757cd9df7c-zrnwd"]
Jul 17 09:49:01.759913 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:01.759877    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" path="/var/lib/kubelet/pods/6e4575b8-3d97-44e4-b609-e76f012e90fe/volumes"
Jul 17 09:49:10.017322 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017292    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"]
Jul 17 09:49:10.017755 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017732    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.017755 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017745    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.017755 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017755    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.017876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017760    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.017876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017772    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.017876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017778    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.017876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017784    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.017876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017789    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.017876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017798    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.017876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017803    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.017876 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017873    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.018139 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017882    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.018139 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.017888    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:49:10.020803 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.020785    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:10.023918 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.023882    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-8f9d7266-serving-cert\""
Jul 17 09:49:10.036525 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.036496    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"]
Jul 17 09:49:10.102842 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.102806    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhnzv\" (UniqueName: \"kubernetes.io/projected/87869771-e021-4fd0-9967-b41ed0b2ebfc-kube-api-access-bhnzv\") pod \"maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:10.102842 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.102843    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87869771-e021-4fd0-9967-b41ed0b2ebfc-openshift-service-ca\") pod \"maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:10.103047 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.102870    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87869771-e021-4fd0-9967-b41ed0b2ebfc-maas-api-tls\") pod \"maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:10.203387 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.203359    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhnzv\" (UniqueName: \"kubernetes.io/projected/87869771-e021-4fd0-9967-b41ed0b2ebfc-kube-api-access-bhnzv\") pod \"maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:10.203540 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.203395    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87869771-e021-4fd0-9967-b41ed0b2ebfc-openshift-service-ca\") pod \"maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:10.203677 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.203655    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87869771-e021-4fd0-9967-b41ed0b2ebfc-maas-api-tls\") pod \"maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:10.204144 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.204120    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87869771-e021-4fd0-9967-b41ed0b2ebfc-openshift-service-ca\") pod \"maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:10.206102 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.206083    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87869771-e021-4fd0-9967-b41ed0b2ebfc-maas-api-tls\") pod \"maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:10.219598 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.219572    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhnzv\" (UniqueName: \"kubernetes.io/projected/87869771-e021-4fd0-9967-b41ed0b2ebfc-kube-api-access-bhnzv\") pod \"maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:10.331973 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.331889    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:10.482725 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.482695    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"]
Jul 17 09:49:10.483162 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:49:10.483137    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87869771_e021_4fd0_9967_b41ed0b2ebfc.slice/crio-6252e69335d64aa36087f3c04f114f83249973adf70f2b03f76acea2519a98c4 WatchSource:0}: Error finding container 6252e69335d64aa36087f3c04f114f83249973adf70f2b03f76acea2519a98c4: Status 404 returned error can't find the container with id 6252e69335d64aa36087f3c04f114f83249973adf70f2b03f76acea2519a98c4
Jul 17 09:49:10.655829 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:10.655720    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" event={"ID":"87869771-e021-4fd0-9967-b41ed0b2ebfc","Type":"ContainerStarted","Data":"6252e69335d64aa36087f3c04f114f83249973adf70f2b03f76acea2519a98c4"}
Jul 17 09:49:11.663454 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:11.663421    2571 generic.go:356] "Generic (PLEG): container finished" podID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerID="9dbff66774dd9cc41dd155521d03c43bdd4a70d6bc572e8bda1b76f1252ab6e6" exitCode=1
Jul 17 09:49:11.663872 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:11.663505    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" event={"ID":"87869771-e021-4fd0-9967-b41ed0b2ebfc","Type":"ContainerDied","Data":"9dbff66774dd9cc41dd155521d03c43bdd4a70d6bc572e8bda1b76f1252ab6e6"}
Jul 17 09:49:11.663872 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:11.663845    2571 scope.go:117] "RemoveContainer" containerID="9dbff66774dd9cc41dd155521d03c43bdd4a70d6bc572e8bda1b76f1252ab6e6"
Jul 17 09:49:12.671433 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:12.671337    2571 generic.go:356] "Generic (PLEG): container finished" podID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerID="1d4776c36e250b963066fca647484611b3642bf585e1a7a73faeaac495f5658b" exitCode=1
Jul 17 09:49:12.671922 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:12.671426    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" event={"ID":"87869771-e021-4fd0-9967-b41ed0b2ebfc","Type":"ContainerDied","Data":"1d4776c36e250b963066fca647484611b3642bf585e1a7a73faeaac495f5658b"}
Jul 17 09:49:12.671922 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:12.671512    2571 scope.go:117] "RemoveContainer" containerID="9dbff66774dd9cc41dd155521d03c43bdd4a70d6bc572e8bda1b76f1252ab6e6"
Jul 17 09:49:12.671922 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:12.671755    2571 scope.go:117] "RemoveContainer" containerID="1d4776c36e250b963066fca647484611b3642bf585e1a7a73faeaac495f5658b"
Jul 17 09:49:12.672085 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:49:12.671984    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz_odh-ai-gateway-infra(87869771-e021-4fd0-9967-b41ed0b2ebfc)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc"
Jul 17 09:49:13.678741 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:13.678696    2571 scope.go:117] "RemoveContainer" containerID="1d4776c36e250b963066fca647484611b3642bf585e1a7a73faeaac495f5658b"
Jul 17 09:49:13.679213 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:49:13.678952    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz_odh-ai-gateway-infra(87869771-e021-4fd0-9967-b41ed0b2ebfc)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc"
Jul 17 09:49:15.332262 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:15.332231    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:15.332667 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:15.332641    2571 scope.go:117] "RemoveContainer" containerID="1d4776c36e250b963066fca647484611b3642bf585e1a7a73faeaac495f5658b"
Jul 17 09:49:15.332840 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:49:15.332819    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz_odh-ai-gateway-infra(87869771-e021-4fd0-9967-b41ed0b2ebfc)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc"
Jul 17 09:49:20.332881 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:20.332847    2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:20.333270 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:20.333232    2571 scope.go:117] "RemoveContainer" containerID="1d4776c36e250b963066fca647484611b3642bf585e1a7a73faeaac495f5658b"
Jul 17 09:49:20.333432 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:49:20.333412    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 10s restarting failed container=maas-api pod=maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz_odh-ai-gateway-infra(87869771-e021-4fd0-9967-b41ed0b2ebfc)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc"
Jul 17 09:49:32.752668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:32.752639    2571 scope.go:117] "RemoveContainer" containerID="1d4776c36e250b963066fca647484611b3642bf585e1a7a73faeaac495f5658b"
Jul 17 09:49:33.798826 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:33.798744    2571 generic.go:356] "Generic (PLEG): container finished" podID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerID="cfeab39b74d520a3f8171ece32ba79538e42ef9567a23d8df65fa8339644e67f" exitCode=1
Jul 17 09:49:33.798826 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:33.798807    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" event={"ID":"87869771-e021-4fd0-9967-b41ed0b2ebfc","Type":"ContainerDied","Data":"cfeab39b74d520a3f8171ece32ba79538e42ef9567a23d8df65fa8339644e67f"}
Jul 17 09:49:33.799245 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:33.798852    2571 scope.go:117] "RemoveContainer" containerID="1d4776c36e250b963066fca647484611b3642bf585e1a7a73faeaac495f5658b"
Jul 17 09:49:33.799245 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:33.799236    2571 scope.go:117] "RemoveContainer" containerID="cfeab39b74d520a3f8171ece32ba79538e42ef9567a23d8df65fa8339644e67f"
Jul 17 09:49:33.799518 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:49:33.799492    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz_odh-ai-gateway-infra(87869771-e021-4fd0-9967-b41ed0b2ebfc)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc"
Jul 17 09:49:35.332421 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:35.332386    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:35.332863 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:35.332820    2571 scope.go:117] "RemoveContainer" containerID="cfeab39b74d520a3f8171ece32ba79538e42ef9567a23d8df65fa8339644e67f"
Jul 17 09:49:35.333026 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:49:35.333006    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz_odh-ai-gateway-infra(87869771-e021-4fd0-9967-b41ed0b2ebfc)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc"
Jul 17 09:49:40.332998 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:40.332947    2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:49:40.333519 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:40.333351    2571 scope.go:117] "RemoveContainer" containerID="cfeab39b74d520a3f8171ece32ba79538e42ef9567a23d8df65fa8339644e67f"
Jul 17 09:49:40.333581 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:49:40.333548    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 20s restarting failed container=maas-api pod=maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz_odh-ai-gateway-infra(87869771-e021-4fd0-9967-b41ed0b2ebfc)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc"
Jul 17 09:49:55.752221 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:55.752186    2571 scope.go:117] "RemoveContainer" containerID="cfeab39b74d520a3f8171ece32ba79538e42ef9567a23d8df65fa8339644e67f"
Jul 17 09:49:56.931918 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:56.931841    2571 generic.go:356] "Generic (PLEG): container finished" podID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerID="c0690eda3dbdb117e33763d525918aa72ee6c8a8df97b244c8508e8606090c1c" exitCode=1
Jul 17 09:49:56.931918 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:56.931880    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" event={"ID":"87869771-e021-4fd0-9967-b41ed0b2ebfc","Type":"ContainerDied","Data":"c0690eda3dbdb117e33763d525918aa72ee6c8a8df97b244c8508e8606090c1c"}
Jul 17 09:49:56.932343 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:56.931928    2571 scope.go:117] "RemoveContainer" containerID="cfeab39b74d520a3f8171ece32ba79538e42ef9567a23d8df65fa8339644e67f"
Jul 17 09:49:56.932343 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:49:56.932329    2571 scope.go:117] "RemoveContainer" containerID="c0690eda3dbdb117e33763d525918aa72ee6c8a8df97b244c8508e8606090c1c"
Jul 17 09:49:56.932578 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:49:56.932553    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz_odh-ai-gateway-infra(87869771-e021-4fd0-9967-b41ed0b2ebfc)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc"
Jul 17 09:50:00.332655 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:00.332567    2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:50:00.332655 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:00.332630    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:50:00.333102 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:00.333015    2571 scope.go:117] "RemoveContainer" containerID="c0690eda3dbdb117e33763d525918aa72ee6c8a8df97b244c8508e8606090c1c"
Jul 17 09:50:00.333214 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:50:00.333192    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz_odh-ai-gateway-infra(87869771-e021-4fd0-9967-b41ed0b2ebfc)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc"
Jul 17 09:50:11.752041 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:11.752006    2571 scope.go:117] "RemoveContainer" containerID="c0690eda3dbdb117e33763d525918aa72ee6c8a8df97b244c8508e8606090c1c"
Jul 17 09:50:11.754722 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:50:11.752258    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CrashLoopBackOff: \"back-off 40s restarting failed container=maas-api pod=maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz_odh-ai-gateway-infra(87869771-e021-4fd0-9967-b41ed0b2ebfc)\"" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc"
Jul 17 09:50:20.045459 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:20.045422    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"]
Jul 17 09:50:20.187598 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:20.187572    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:50:20.232822 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:20.232785    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87869771-e021-4fd0-9967-b41ed0b2ebfc-openshift-service-ca\") pod \"87869771-e021-4fd0-9967-b41ed0b2ebfc\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") "
Jul 17 09:50:20.232822 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:20.232829    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87869771-e021-4fd0-9967-b41ed0b2ebfc-maas-api-tls\") pod \"87869771-e021-4fd0-9967-b41ed0b2ebfc\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") "
Jul 17 09:50:20.233065 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:20.232883    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhnzv\" (UniqueName: \"kubernetes.io/projected/87869771-e021-4fd0-9967-b41ed0b2ebfc-kube-api-access-bhnzv\") pod \"87869771-e021-4fd0-9967-b41ed0b2ebfc\" (UID: \"87869771-e021-4fd0-9967-b41ed0b2ebfc\") "
Jul 17 09:50:20.233132 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:20.233115    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87869771-e021-4fd0-9967-b41ed0b2ebfc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87869771-e021-4fd0-9967-b41ed0b2ebfc" (UID: "87869771-e021-4fd0-9967-b41ed0b2ebfc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:50:20.235080 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:20.235046    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87869771-e021-4fd0-9967-b41ed0b2ebfc-kube-api-access-bhnzv" (OuterVolumeSpecName: "kube-api-access-bhnzv") pod "87869771-e021-4fd0-9967-b41ed0b2ebfc" (UID: "87869771-e021-4fd0-9967-b41ed0b2ebfc"). InnerVolumeSpecName "kube-api-access-bhnzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:50:20.235080 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:20.235052    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87869771-e021-4fd0-9967-b41ed0b2ebfc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87869771-e021-4fd0-9967-b41ed0b2ebfc" (UID: "87869771-e021-4fd0-9967-b41ed0b2ebfc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:50:20.333761 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:20.333677    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87869771-e021-4fd0-9967-b41ed0b2ebfc-openshift-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:50:20.333761 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:20.333706    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87869771-e021-4fd0-9967-b41ed0b2ebfc-maas-api-tls\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:50:20.333761 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:20.333716    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhnzv\" (UniqueName: \"kubernetes.io/projected/87869771-e021-4fd0-9967-b41ed0b2ebfc-kube-api-access-bhnzv\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:50:21.072274 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:21.072236    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz" event={"ID":"87869771-e021-4fd0-9967-b41ed0b2ebfc","Type":"ContainerDied","Data":"6252e69335d64aa36087f3c04f114f83249973adf70f2b03f76acea2519a98c4"}
Jul 17 09:50:21.072842 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:21.072816    2571 scope.go:117] "RemoveContainer" containerID="c0690eda3dbdb117e33763d525918aa72ee6c8a8df97b244c8508e8606090c1c"
Jul 17 09:50:21.072964 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:21.072290    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"
Jul 17 09:50:21.112490 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:21.112439    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"]
Jul 17 09:50:21.116992 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:21.116963    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-8f9d7266-5dcb6f5f6f-gvklz"]
Jul 17 09:50:21.758450 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:50:21.758407    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" path="/var/lib/kubelet/pods/87869771-e021-4fd0-9967-b41ed0b2ebfc/volumes"
Jul 17 09:51:04.382792 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.382748    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"]
Jul 17 09:51:04.383417 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.383395    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:51:04.383532 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.383419    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:51:04.383532 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.383431    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:04.383532 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.383438    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:04.383532 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.383493    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:04.383532 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.383503    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:04.383807 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.383643    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:51:04.383807 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.383657    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:51:04.383807 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.383665    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:04.383807 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.383674    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:04.383807 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.383684    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:04.387043 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.387022    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"
Jul 17 09:51:04.390133 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.390116    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:51:04.395915 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.395888    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"]
Jul 17 09:51:04.424224 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.424182    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4244d87-6025-4f8e-8d3c-cd4afba4ab27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j\" (UID: \"d4244d87-6025-4f8e-8d3c-cd4afba4ab27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"
Jul 17 09:51:04.525381 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.525347    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4244d87-6025-4f8e-8d3c-cd4afba4ab27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j\" (UID: \"d4244d87-6025-4f8e-8d3c-cd4afba4ab27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"
Jul 17 09:51:04.525998 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.525973    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4244d87-6025-4f8e-8d3c-cd4afba4ab27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j\" (UID: \"d4244d87-6025-4f8e-8d3c-cd4afba4ab27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"
Jul 17 09:51:04.699085 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.699049    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"
Jul 17 09:51:04.840329 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:04.840297    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"]
Jul 17 09:51:05.335263 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:05.335166    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j" event={"ID":"d4244d87-6025-4f8e-8d3c-cd4afba4ab27","Type":"ContainerStarted","Data":"25d6f1677f503547df864c373ceed2f9539503ec89d35773324fa8e7e7310e9f"}
Jul 17 09:51:05.335263 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:05.335222    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j" event={"ID":"d4244d87-6025-4f8e-8d3c-cd4afba4ab27","Type":"ContainerStarted","Data":"3832f40e5de55e930dcf535268bf8919e310c5d6aa66c88f4625b9722913f2a6"}
Jul 17 09:51:05.353609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:05.353550    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j" podStartSLOduration=1.3535344839999999 podStartE2EDuration="1.353534484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:51:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:51:05.351903902 +0000 UTC m=+1949.416724625" watchObservedRunningTime="2026-07-17 09:51:05.353534484 +0000 UTC m=+1949.418355205"
Jul 17 09:51:06.348282 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:06.348233    2571 generic.go:356] "Generic (PLEG): container finished" podID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerID="25d6f1677f503547df864c373ceed2f9539503ec89d35773324fa8e7e7310e9f" exitCode=6
Jul 17 09:51:06.348699 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:06.348321    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j" event={"ID":"d4244d87-6025-4f8e-8d3c-cd4afba4ab27","Type":"ContainerDied","Data":"25d6f1677f503547df864c373ceed2f9539503ec89d35773324fa8e7e7310e9f"}
Jul 17 09:51:06.348699 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:06.348522    2571 scope.go:117] "RemoveContainer" containerID="25d6f1677f503547df864c373ceed2f9539503ec89d35773324fa8e7e7310e9f"
Jul 17 09:51:07.356419 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:07.356328    2571 generic.go:356] "Generic (PLEG): container finished" podID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerID="c2d15aa73b7ea1f673e9e0283723944ac2d3e67245514aa280a24882d7eed8ab" exitCode=6
Jul 17 09:51:07.356841 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:07.356408    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j" event={"ID":"d4244d87-6025-4f8e-8d3c-cd4afba4ab27","Type":"ContainerDied","Data":"c2d15aa73b7ea1f673e9e0283723944ac2d3e67245514aa280a24882d7eed8ab"}
Jul 17 09:51:07.356841 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:07.356485    2571 scope.go:117] "RemoveContainer" containerID="25d6f1677f503547df864c373ceed2f9539503ec89d35773324fa8e7e7310e9f"
Jul 17 09:51:07.356841 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:07.356618    2571 scope.go:117] "RemoveContainer" containerID="c2d15aa73b7ea1f673e9e0283723944ac2d3e67245514aa280a24882d7eed8ab"
Jul 17 09:51:07.356841 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:51:07.356818    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j_odh-ai-gateway-infra(d4244d87-6025-4f8e-8d3c-cd4afba4ab27)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27"
Jul 17 09:51:08.363800 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:08.363770    2571 scope.go:117] "RemoveContainer" containerID="c2d15aa73b7ea1f673e9e0283723944ac2d3e67245514aa280a24882d7eed8ab"
Jul 17 09:51:08.364175 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:51:08.363991    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j_odh-ai-gateway-infra(d4244d87-6025-4f8e-8d3c-cd4afba4ab27)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27"
Jul 17 09:51:22.752403 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:22.752371    2571 scope.go:117] "RemoveContainer" containerID="c2d15aa73b7ea1f673e9e0283723944ac2d3e67245514aa280a24882d7eed8ab"
Jul 17 09:51:23.402165 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.402129    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"]
Jul 17 09:51:23.402597 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.402578    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:23.402597 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.402594    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:23.402753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.402617    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:23.402753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.402622    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:23.402753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.402700    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e4575b8-3d97-44e4-b609-e76f012e90fe" containerName="maas-api"
Jul 17 09:51:23.402753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.402710    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="87869771-e021-4fd0-9967-b41ed0b2ebfc" containerName="maas-api"
Jul 17 09:51:23.406164 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.406145    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"
Jul 17 09:51:23.414557 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.414533    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"]
Jul 17 09:51:23.448002 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.447974    2571 generic.go:356] "Generic (PLEG): container finished" podID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerID="f03ae8641b942a1444a1fb05326309b74c4fc275ca1664a69fbcdd6a89bcce8c" exitCode=6
Jul 17 09:51:23.448139 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.448050    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j" event={"ID":"d4244d87-6025-4f8e-8d3c-cd4afba4ab27","Type":"ContainerDied","Data":"f03ae8641b942a1444a1fb05326309b74c4fc275ca1664a69fbcdd6a89bcce8c"}
Jul 17 09:51:23.448139 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.448113    2571 scope.go:117] "RemoveContainer" containerID="c2d15aa73b7ea1f673e9e0283723944ac2d3e67245514aa280a24882d7eed8ab"
Jul 17 09:51:23.448425 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.448402    2571 scope.go:117] "RemoveContainer" containerID="f03ae8641b942a1444a1fb05326309b74c4fc275ca1664a69fbcdd6a89bcce8c"
Jul 17 09:51:23.448672 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:51:23.448650    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j_odh-ai-gateway-infra(d4244d87-6025-4f8e-8d3c-cd4afba4ab27)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27"
Jul 17 09:51:23.501039 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.500985    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29adb6a8-5ae0-4688-9b87-811d1171bda8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn\" (UID: \"29adb6a8-5ae0-4688-9b87-811d1171bda8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"
Jul 17 09:51:23.602310 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.602284    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29adb6a8-5ae0-4688-9b87-811d1171bda8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn\" (UID: \"29adb6a8-5ae0-4688-9b87-811d1171bda8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"
Jul 17 09:51:23.602905 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.602888    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29adb6a8-5ae0-4688-9b87-811d1171bda8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn\" (UID: \"29adb6a8-5ae0-4688-9b87-811d1171bda8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"
Jul 17 09:51:23.719021 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.718991    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"
Jul 17 09:51:23.858825 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:23.858779    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"]
Jul 17 09:51:23.859808 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:51:23.859780    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29adb6a8_5ae0_4688_9b87_811d1171bda8.slice/crio-f08b72776be2cae35249735f33e11abcd25576c18fbf3f2d8404b77ba98f79f7 WatchSource:0}: Error finding container f08b72776be2cae35249735f33e11abcd25576c18fbf3f2d8404b77ba98f79f7: Status 404 returned error can't find the container with id f08b72776be2cae35249735f33e11abcd25576c18fbf3f2d8404b77ba98f79f7
Jul 17 09:51:24.455302 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:24.455263    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn" event={"ID":"29adb6a8-5ae0-4688-9b87-811d1171bda8","Type":"ContainerStarted","Data":"db6bcd79ba39d1dd33f9f96415ab2c89f0a977dfd4fba8da8445872d4c6b86fa"}
Jul 17 09:51:24.455498 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:24.455349    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn" event={"ID":"29adb6a8-5ae0-4688-9b87-811d1171bda8","Type":"ContainerStarted","Data":"f08b72776be2cae35249735f33e11abcd25576c18fbf3f2d8404b77ba98f79f7"}
Jul 17 09:51:24.474360 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:24.474314    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn" podStartSLOduration=1.4742986949999999 podStartE2EDuration="1.474298695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:51:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:51:24.472792851 +0000 UTC m=+1968.537613576" watchObservedRunningTime="2026-07-17 09:51:24.474298695 +0000 UTC m=+1968.539119417"
Jul 17 09:51:24.478481 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:24.478447    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"]
Jul 17 09:51:24.631622 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:24.631601    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"
Jul 17 09:51:24.711867 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:24.711803    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4244d87-6025-4f8e-8d3c-cd4afba4ab27-maas-api-service-ca\") pod \"d4244d87-6025-4f8e-8d3c-cd4afba4ab27\" (UID: \"d4244d87-6025-4f8e-8d3c-cd4afba4ab27\") "
Jul 17 09:51:24.712168 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:24.712146    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4244d87-6025-4f8e-8d3c-cd4afba4ab27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d4244d87-6025-4f8e-8d3c-cd4afba4ab27" (UID: "d4244d87-6025-4f8e-8d3c-cd4afba4ab27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:51:24.813219 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:24.813191    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4244d87-6025-4f8e-8d3c-cd4afba4ab27-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:51:25.463759 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:25.463714    2571 generic.go:356] "Generic (PLEG): container finished" podID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerID="db6bcd79ba39d1dd33f9f96415ab2c89f0a977dfd4fba8da8445872d4c6b86fa" exitCode=7
Jul 17 09:51:25.464153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:25.463796    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn" event={"ID":"29adb6a8-5ae0-4688-9b87-811d1171bda8","Type":"ContainerDied","Data":"db6bcd79ba39d1dd33f9f96415ab2c89f0a977dfd4fba8da8445872d4c6b86fa"}
Jul 17 09:51:25.464153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:25.464089    2571 scope.go:117] "RemoveContainer" containerID="db6bcd79ba39d1dd33f9f96415ab2c89f0a977dfd4fba8da8445872d4c6b86fa"
Jul 17 09:51:25.465251 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:25.465224    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j" event={"ID":"d4244d87-6025-4f8e-8d3c-cd4afba4ab27","Type":"ContainerDied","Data":"3832f40e5de55e930dcf535268bf8919e310c5d6aa66c88f4625b9722913f2a6"}
Jul 17 09:51:25.465355 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:25.465274    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"
Jul 17 09:51:25.465355 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:25.465277    2571 scope.go:117] "RemoveContainer" containerID="f03ae8641b942a1444a1fb05326309b74c4fc275ca1664a69fbcdd6a89bcce8c"
Jul 17 09:51:25.498602 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:25.498566    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"]
Jul 17 09:51:25.502616 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:25.502582    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cnm8j"]
Jul 17 09:51:25.758714 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:25.758685    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" path="/var/lib/kubelet/pods/d4244d87-6025-4f8e-8d3c-cd4afba4ab27/volumes"
Jul 17 09:51:26.473077 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:26.473032    2571 generic.go:356] "Generic (PLEG): container finished" podID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerID="a2629e59eb6b4391ae28168774a5c45d40b8686621720be9178a2f4e46f55ac9" exitCode=7
Jul 17 09:51:26.473572 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:26.473117    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn" event={"ID":"29adb6a8-5ae0-4688-9b87-811d1171bda8","Type":"ContainerDied","Data":"a2629e59eb6b4391ae28168774a5c45d40b8686621720be9178a2f4e46f55ac9"}
Jul 17 09:51:26.473572 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:26.473181    2571 scope.go:117] "RemoveContainer" containerID="db6bcd79ba39d1dd33f9f96415ab2c89f0a977dfd4fba8da8445872d4c6b86fa"
Jul 17 09:51:26.473572 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:26.473369    2571 scope.go:117] "RemoveContainer" containerID="a2629e59eb6b4391ae28168774a5c45d40b8686621720be9178a2f4e46f55ac9"
Jul 17 09:51:26.473735 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:51:26.473682    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn_odh-ai-gateway-infra(29adb6a8-5ae0-4688-9b87-811d1171bda8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8"
Jul 17 09:51:27.481332 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:27.481302    2571 scope.go:117] "RemoveContainer" containerID="a2629e59eb6b4391ae28168774a5c45d40b8686621720be9178a2f4e46f55ac9"
Jul 17 09:51:27.481785 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:51:27.481590    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn_odh-ai-gateway-infra(29adb6a8-5ae0-4688-9b87-811d1171bda8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8"
Jul 17 09:51:34.732265 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.732231    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"]
Jul 17 09:51:34.732697 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.732683    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerName="revoke-keys"
Jul 17 09:51:34.732697 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.732698    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerName="revoke-keys"
Jul 17 09:51:34.732776 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.732712    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerName="revoke-keys"
Jul 17 09:51:34.732776 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.732718    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerName="revoke-keys"
Jul 17 09:51:34.732776 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.732725    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerName="revoke-keys"
Jul 17 09:51:34.732776 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.732730    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerName="revoke-keys"
Jul 17 09:51:34.732892 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.732793    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerName="revoke-keys"
Jul 17 09:51:34.732892 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.732802    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerName="revoke-keys"
Jul 17 09:51:34.738490 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.738423    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"
Jul 17 09:51:34.745963 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.745930    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"]
Jul 17 09:51:34.820758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.820705    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec1aaf1-9102-4cad-aca8-c5a5b55f5c89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6\" (UID: \"aec1aaf1-9102-4cad-aca8-c5a5b55f5c89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"
Jul 17 09:51:34.922073 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.922041    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec1aaf1-9102-4cad-aca8-c5a5b55f5c89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6\" (UID: \"aec1aaf1-9102-4cad-aca8-c5a5b55f5c89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"
Jul 17 09:51:34.922813 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:34.922790    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec1aaf1-9102-4cad-aca8-c5a5b55f5c89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6\" (UID: \"aec1aaf1-9102-4cad-aca8-c5a5b55f5c89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"
Jul 17 09:51:35.052143 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:35.052051    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"
Jul 17 09:51:35.195866 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:35.195832    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"]
Jul 17 09:51:35.199415 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:51:35.199386    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec1aaf1_9102_4cad_aca8_c5a5b55f5c89.slice/crio-797dd5bfb019cf99a1f76b5bfda8f5dc96b4cf912368410b5b0d8047f227a168 WatchSource:0}: Error finding container 797dd5bfb019cf99a1f76b5bfda8f5dc96b4cf912368410b5b0d8047f227a168: Status 404 returned error can't find the container with id 797dd5bfb019cf99a1f76b5bfda8f5dc96b4cf912368410b5b0d8047f227a168
Jul 17 09:51:35.530578 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:35.530537    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6" event={"ID":"aec1aaf1-9102-4cad-aca8-c5a5b55f5c89","Type":"ContainerStarted","Data":"797dd5bfb019cf99a1f76b5bfda8f5dc96b4cf912368410b5b0d8047f227a168"}
Jul 17 09:51:36.537907 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:36.537866    2571 generic.go:356] "Generic (PLEG): container finished" podID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerID="82b92ed1c0d1bb4d57f5d96ccdd20a1962b0700674eb34c451ab5d75503a667a" exitCode=6
Jul 17 09:51:36.538314 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:36.537914    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6" event={"ID":"aec1aaf1-9102-4cad-aca8-c5a5b55f5c89","Type":"ContainerDied","Data":"82b92ed1c0d1bb4d57f5d96ccdd20a1962b0700674eb34c451ab5d75503a667a"}
Jul 17 09:51:36.538314 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:36.538142    2571 scope.go:117] "RemoveContainer" containerID="82b92ed1c0d1bb4d57f5d96ccdd20a1962b0700674eb34c451ab5d75503a667a"
Jul 17 09:51:37.546855 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:37.546820    2571 generic.go:356] "Generic (PLEG): container finished" podID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerID="16a10963f4d95b2feef234461d3b675ac8b413757595013cf5edfdd8efac1415" exitCode=6
Jul 17 09:51:37.547272 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:37.546902    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6" event={"ID":"aec1aaf1-9102-4cad-aca8-c5a5b55f5c89","Type":"ContainerDied","Data":"16a10963f4d95b2feef234461d3b675ac8b413757595013cf5edfdd8efac1415"}
Jul 17 09:51:37.547272 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:37.546958    2571 scope.go:117] "RemoveContainer" containerID="82b92ed1c0d1bb4d57f5d96ccdd20a1962b0700674eb34c451ab5d75503a667a"
Jul 17 09:51:37.547272 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:37.547098    2571 scope.go:117] "RemoveContainer" containerID="16a10963f4d95b2feef234461d3b675ac8b413757595013cf5edfdd8efac1415"
Jul 17 09:51:37.547420 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:51:37.547347    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6_odh-ai-gateway-infra(aec1aaf1-9102-4cad-aca8-c5a5b55f5c89)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89"
Jul 17 09:51:38.554554 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:38.554523    2571 scope.go:117] "RemoveContainer" containerID="16a10963f4d95b2feef234461d3b675ac8b413757595013cf5edfdd8efac1415"
Jul 17 09:51:38.554916 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:51:38.554738    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6_odh-ai-gateway-infra(aec1aaf1-9102-4cad-aca8-c5a5b55f5c89)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89"
Jul 17 09:51:39.753146 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:39.753111    2571 scope.go:117] "RemoveContainer" containerID="a2629e59eb6b4391ae28168774a5c45d40b8686621720be9178a2f4e46f55ac9"
Jul 17 09:51:40.569509 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:40.569092    2571 generic.go:356] "Generic (PLEG): container finished" podID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerID="f2f96f0f385399e097b844bb7a29a10e2cf12d779809b9f727fa249f7fd9aa81" exitCode=7
Jul 17 09:51:40.569509 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:40.569296    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn" event={"ID":"29adb6a8-5ae0-4688-9b87-811d1171bda8","Type":"ContainerDied","Data":"f2f96f0f385399e097b844bb7a29a10e2cf12d779809b9f727fa249f7fd9aa81"}
Jul 17 09:51:40.569509 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:40.569354    2571 scope.go:117] "RemoveContainer" containerID="a2629e59eb6b4391ae28168774a5c45d40b8686621720be9178a2f4e46f55ac9"
Jul 17 09:51:40.570583 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:40.569790    2571 scope.go:117] "RemoveContainer" containerID="f2f96f0f385399e097b844bb7a29a10e2cf12d779809b9f727fa249f7fd9aa81"
Jul 17 09:51:40.570583 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:51:40.570047    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn_odh-ai-gateway-infra(29adb6a8-5ae0-4688-9b87-811d1171bda8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8"
Jul 17 09:51:41.618629 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:41.618593    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"]
Jul 17 09:51:41.763874 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:41.763846    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"
Jul 17 09:51:41.888348 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:41.888254    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29adb6a8-5ae0-4688-9b87-811d1171bda8-maas-api-service-ca\") pod \"29adb6a8-5ae0-4688-9b87-811d1171bda8\" (UID: \"29adb6a8-5ae0-4688-9b87-811d1171bda8\") "
Jul 17 09:51:41.888690 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:41.888664    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29adb6a8-5ae0-4688-9b87-811d1171bda8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29adb6a8-5ae0-4688-9b87-811d1171bda8" (UID: "29adb6a8-5ae0-4688-9b87-811d1171bda8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:51:41.989091 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:41.989050    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29adb6a8-5ae0-4688-9b87-811d1171bda8-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:51:42.585408 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:42.585372    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn" event={"ID":"29adb6a8-5ae0-4688-9b87-811d1171bda8","Type":"ContainerDied","Data":"f08b72776be2cae35249735f33e11abcd25576c18fbf3f2d8404b77ba98f79f7"}
Jul 17 09:51:42.585408 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:42.585395    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"
Jul 17 09:51:42.585700 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:42.585437    2571 scope.go:117] "RemoveContainer" containerID="f2f96f0f385399e097b844bb7a29a10e2cf12d779809b9f727fa249f7fd9aa81"
Jul 17 09:51:42.624576 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:42.624536    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"]
Jul 17 09:51:42.627171 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:42.627139    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-2j2dn"]
Jul 17 09:51:43.758989 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:43.758952    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8" path="/var/lib/kubelet/pods/29adb6a8-5ae0-4688-9b87-811d1171bda8/volumes"
Jul 17 09:51:52.752730 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:52.752696    2571 scope.go:117] "RemoveContainer" containerID="16a10963f4d95b2feef234461d3b675ac8b413757595013cf5edfdd8efac1415"
Jul 17 09:51:53.652078 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:53.651989    2571 generic.go:356] "Generic (PLEG): container finished" podID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerID="47392ceba39a0a00a7d4ea666e4301b03a6622a3a2e5ceea7fe98a2a5165d6ba" exitCode=6
Jul 17 09:51:53.652078 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:53.652064    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6" event={"ID":"aec1aaf1-9102-4cad-aca8-c5a5b55f5c89","Type":"ContainerDied","Data":"47392ceba39a0a00a7d4ea666e4301b03a6622a3a2e5ceea7fe98a2a5165d6ba"}
Jul 17 09:51:53.652279 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:53.652124    2571 scope.go:117] "RemoveContainer" containerID="16a10963f4d95b2feef234461d3b675ac8b413757595013cf5edfdd8efac1415"
Jul 17 09:51:53.652437 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:53.652419    2571 scope.go:117] "RemoveContainer" containerID="47392ceba39a0a00a7d4ea666e4301b03a6622a3a2e5ceea7fe98a2a5165d6ba"
Jul 17 09:51:53.652679 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:51:53.652657    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6_odh-ai-gateway-infra(aec1aaf1-9102-4cad-aca8-c5a5b55f5c89)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89"
Jul 17 09:51:54.702414 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:54.696988    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"]
Jul 17 09:51:54.850410 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:54.850386    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"
Jul 17 09:51:54.905454 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:54.905421    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec1aaf1-9102-4cad-aca8-c5a5b55f5c89-maas-api-service-ca\") pod \"aec1aaf1-9102-4cad-aca8-c5a5b55f5c89\" (UID: \"aec1aaf1-9102-4cad-aca8-c5a5b55f5c89\") "
Jul 17 09:51:54.905783 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:54.905760    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aec1aaf1-9102-4cad-aca8-c5a5b55f5c89-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" (UID: "aec1aaf1-9102-4cad-aca8-c5a5b55f5c89"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:51:55.007024 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:55.006920    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aec1aaf1-9102-4cad-aca8-c5a5b55f5c89-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:51:55.665973 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:55.665941    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"
Jul 17 09:51:55.666172 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:55.665985    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6" event={"ID":"aec1aaf1-9102-4cad-aca8-c5a5b55f5c89","Type":"ContainerDied","Data":"797dd5bfb019cf99a1f76b5bfda8f5dc96b4cf912368410b5b0d8047f227a168"}
Jul 17 09:51:55.666172 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:55.666053    2571 scope.go:117] "RemoveContainer" containerID="47392ceba39a0a00a7d4ea666e4301b03a6622a3a2e5ceea7fe98a2a5165d6ba"
Jul 17 09:51:55.697184 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:55.697159    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"]
Jul 17 09:51:55.703320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:55.703292    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-s4lr6"]
Jul 17 09:51:55.758584 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:51:55.758554    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" path="/var/lib/kubelet/pods/aec1aaf1-9102-4cad-aca8-c5a5b55f5c89/volumes"
Jul 17 09:52:04.467226 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.467187    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"]
Jul 17 09:52:04.467841 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.467820    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerName="revoke-keys"
Jul 17 09:52:04.467930 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.467842    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerName="revoke-keys"
Jul 17 09:52:04.467930 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.467863    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerName="revoke-keys"
Jul 17 09:52:04.467930 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.467871    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerName="revoke-keys"
Jul 17 09:52:04.467930 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.467908    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerName="revoke-keys"
Jul 17 09:52:04.467930 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.467917    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerName="revoke-keys"
Jul 17 09:52:04.468177 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.467936    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerName="revoke-keys"
Jul 17 09:52:04.468177 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.467944    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerName="revoke-keys"
Jul 17 09:52:04.468177 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.468040    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4244d87-6025-4f8e-8d3c-cd4afba4ab27" containerName="revoke-keys"
Jul 17 09:52:04.468177 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.468052    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerName="revoke-keys"
Jul 17 09:52:04.468177 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.468064    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerName="revoke-keys"
Jul 17 09:52:04.468177 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.468074    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerName="revoke-keys"
Jul 17 09:52:04.468177 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.468083    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerName="revoke-keys"
Jul 17 09:52:04.468177 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.468093    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerName="revoke-keys"
Jul 17 09:52:04.472785 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.472763    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"
Jul 17 09:52:04.476744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.476725    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:52:04.490165 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.490134    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"]
Jul 17 09:52:04.598851 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.598811    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc2b451-62aa-4cde-9af3-246f1f8462f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht\" (UID: \"0bc2b451-62aa-4cde-9af3-246f1f8462f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"
Jul 17 09:52:04.699508 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.699444    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc2b451-62aa-4cde-9af3-246f1f8462f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht\" (UID: \"0bc2b451-62aa-4cde-9af3-246f1f8462f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"
Jul 17 09:52:04.700083 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.700063    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc2b451-62aa-4cde-9af3-246f1f8462f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht\" (UID: \"0bc2b451-62aa-4cde-9af3-246f1f8462f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"
Jul 17 09:52:04.783426 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.783333    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"
Jul 17 09:52:04.925687 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:04.925658    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"]
Jul 17 09:52:04.926743 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:52:04.926712    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc2b451_62aa_4cde_9af3_246f1f8462f9.slice/crio-de81e67315a652dda2fbfa434c41e35857a3f1500887f877e723d69a35f45231 WatchSource:0}: Error finding container de81e67315a652dda2fbfa434c41e35857a3f1500887f877e723d69a35f45231: Status 404 returned error can't find the container with id de81e67315a652dda2fbfa434c41e35857a3f1500887f877e723d69a35f45231
Jul 17 09:52:05.725920 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:05.725886    2571 generic.go:356] "Generic (PLEG): container finished" podID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerID="b12320238eb91efbeab409dfced8c6db98f4ed698bb9cf86f49f8d839565068f" exitCode=6
Jul 17 09:52:05.726288 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:05.725945    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht" event={"ID":"0bc2b451-62aa-4cde-9af3-246f1f8462f9","Type":"ContainerDied","Data":"b12320238eb91efbeab409dfced8c6db98f4ed698bb9cf86f49f8d839565068f"}
Jul 17 09:52:05.726288 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:05.725986    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht" event={"ID":"0bc2b451-62aa-4cde-9af3-246f1f8462f9","Type":"ContainerStarted","Data":"de81e67315a652dda2fbfa434c41e35857a3f1500887f877e723d69a35f45231"}
Jul 17 09:52:05.726288 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:05.726168    2571 scope.go:117] "RemoveContainer" containerID="b12320238eb91efbeab409dfced8c6db98f4ed698bb9cf86f49f8d839565068f"
Jul 17 09:52:06.733645 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:06.733611    2571 generic.go:356] "Generic (PLEG): container finished" podID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerID="854f0ed9346cd14d4c3a528bae5a27e47ca68f2b9e7fa115148753edd301250c" exitCode=6
Jul 17 09:52:06.734054 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:06.733700    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht" event={"ID":"0bc2b451-62aa-4cde-9af3-246f1f8462f9","Type":"ContainerDied","Data":"854f0ed9346cd14d4c3a528bae5a27e47ca68f2b9e7fa115148753edd301250c"}
Jul 17 09:52:06.734054 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:06.733770    2571 scope.go:117] "RemoveContainer" containerID="b12320238eb91efbeab409dfced8c6db98f4ed698bb9cf86f49f8d839565068f"
Jul 17 09:52:06.734054 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:06.733944    2571 scope.go:117] "RemoveContainer" containerID="854f0ed9346cd14d4c3a528bae5a27e47ca68f2b9e7fa115148753edd301250c"
Jul 17 09:52:06.734195 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:52:06.734172    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht_odh-ai-gateway-infra(0bc2b451-62aa-4cde-9af3-246f1f8462f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9"
Jul 17 09:52:07.740929 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:07.740900    2571 scope.go:117] "RemoveContainer" containerID="854f0ed9346cd14d4c3a528bae5a27e47ca68f2b9e7fa115148753edd301250c"
Jul 17 09:52:07.741328 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:52:07.741115    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht_odh-ai-gateway-infra(0bc2b451-62aa-4cde-9af3-246f1f8462f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9"
Jul 17 09:52:13.468579 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.468535    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"]
Jul 17 09:52:13.469204 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.468986    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerName="revoke-keys"
Jul 17 09:52:13.469204 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.468998    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="29adb6a8-5ae0-4688-9b87-811d1171bda8" containerName="revoke-keys"
Jul 17 09:52:13.469204 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.469026    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerName="revoke-keys"
Jul 17 09:52:13.469204 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.469032    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerName="revoke-keys"
Jul 17 09:52:13.469204 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.469108    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec1aaf1-9102-4cad-aca8-c5a5b55f5c89" containerName="revoke-keys"
Jul 17 09:52:13.472007 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.471990    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"
Jul 17 09:52:13.487837 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.487796    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"]
Jul 17 09:52:13.579278 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.579241    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d20ae0-0985-4d25-888e-6bdcdb9ac659-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm\" (UID: \"18d20ae0-0985-4d25-888e-6bdcdb9ac659\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"
Jul 17 09:52:13.680780 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.680741    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d20ae0-0985-4d25-888e-6bdcdb9ac659-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm\" (UID: \"18d20ae0-0985-4d25-888e-6bdcdb9ac659\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"
Jul 17 09:52:13.681434 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.681405    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d20ae0-0985-4d25-888e-6bdcdb9ac659-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm\" (UID: \"18d20ae0-0985-4d25-888e-6bdcdb9ac659\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"
Jul 17 09:52:13.782397 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.782318    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"
Jul 17 09:52:13.937211 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:13.937187    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"]
Jul 17 09:52:13.939534 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:52:13.939507    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18d20ae0_0985_4d25_888e_6bdcdb9ac659.slice/crio-e2698fb3b8af7f46e5620c438e3529059eea0ebb9c2d9e3b08ae93864bd797b6 WatchSource:0}: Error finding container e2698fb3b8af7f46e5620c438e3529059eea0ebb9c2d9e3b08ae93864bd797b6: Status 404 returned error can't find the container with id e2698fb3b8af7f46e5620c438e3529059eea0ebb9c2d9e3b08ae93864bd797b6
Jul 17 09:52:14.782281 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:14.782199    2571 generic.go:356] "Generic (PLEG): container finished" podID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerID="501cdd868417136690ee9ffa95c5e48535fb3b208f6ce7b9e7c0117b66fd86e6" exitCode=7
Jul 17 09:52:14.782680 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:14.782282    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm" event={"ID":"18d20ae0-0985-4d25-888e-6bdcdb9ac659","Type":"ContainerDied","Data":"501cdd868417136690ee9ffa95c5e48535fb3b208f6ce7b9e7c0117b66fd86e6"}
Jul 17 09:52:14.782680 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:14.782332    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm" event={"ID":"18d20ae0-0985-4d25-888e-6bdcdb9ac659","Type":"ContainerStarted","Data":"e2698fb3b8af7f46e5620c438e3529059eea0ebb9c2d9e3b08ae93864bd797b6"}
Jul 17 09:52:14.782680 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:14.782494    2571 scope.go:117] "RemoveContainer" containerID="501cdd868417136690ee9ffa95c5e48535fb3b208f6ce7b9e7c0117b66fd86e6"
Jul 17 09:52:15.789425 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:15.789382    2571 generic.go:356] "Generic (PLEG): container finished" podID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerID="638bc6b1b824ac5b14436716916e9f3dd29b97061771a320b4b98392456cbaf0" exitCode=7
Jul 17 09:52:15.789821 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:15.789450    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm" event={"ID":"18d20ae0-0985-4d25-888e-6bdcdb9ac659","Type":"ContainerDied","Data":"638bc6b1b824ac5b14436716916e9f3dd29b97061771a320b4b98392456cbaf0"}
Jul 17 09:52:15.789821 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:15.789555    2571 scope.go:117] "RemoveContainer" containerID="501cdd868417136690ee9ffa95c5e48535fb3b208f6ce7b9e7c0117b66fd86e6"
Jul 17 09:52:15.789821 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:15.789819    2571 scope.go:117] "RemoveContainer" containerID="638bc6b1b824ac5b14436716916e9f3dd29b97061771a320b4b98392456cbaf0"
Jul 17 09:52:15.790052 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:52:15.790031    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm_odh-ai-gateway-infra(18d20ae0-0985-4d25-888e-6bdcdb9ac659)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659"
Jul 17 09:52:16.796393 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:16.796367    2571 scope.go:117] "RemoveContainer" containerID="638bc6b1b824ac5b14436716916e9f3dd29b97061771a320b4b98392456cbaf0"
Jul 17 09:52:16.796777 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:52:16.796580    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm_odh-ai-gateway-infra(18d20ae0-0985-4d25-888e-6bdcdb9ac659)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659"
Jul 17 09:52:21.752146 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:21.752111    2571 scope.go:117] "RemoveContainer" containerID="854f0ed9346cd14d4c3a528bae5a27e47ca68f2b9e7fa115148753edd301250c"
Jul 17 09:52:22.831643 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:22.831609    2571 generic.go:356] "Generic (PLEG): container finished" podID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerID="0fca05632ccbc8900254396074e062180b581bd277d9fcfdbb34c126b63eaff4" exitCode=6
Jul 17 09:52:22.832091 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:22.831661    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht" event={"ID":"0bc2b451-62aa-4cde-9af3-246f1f8462f9","Type":"ContainerDied","Data":"0fca05632ccbc8900254396074e062180b581bd277d9fcfdbb34c126b63eaff4"}
Jul 17 09:52:22.832091 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:22.831712    2571 scope.go:117] "RemoveContainer" containerID="854f0ed9346cd14d4c3a528bae5a27e47ca68f2b9e7fa115148753edd301250c"
Jul 17 09:52:22.832091 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:22.832003    2571 scope.go:117] "RemoveContainer" containerID="0fca05632ccbc8900254396074e062180b581bd277d9fcfdbb34c126b63eaff4"
Jul 17 09:52:22.832298 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:52:22.832274    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht_odh-ai-gateway-infra(0bc2b451-62aa-4cde-9af3-246f1f8462f9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9"
Jul 17 09:52:23.886176 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:23.886137    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"]
Jul 17 09:52:24.036713 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:24.036684    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"
Jul 17 09:52:24.175683 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:24.175594    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc2b451-62aa-4cde-9af3-246f1f8462f9-maas-api-service-ca\") pod \"0bc2b451-62aa-4cde-9af3-246f1f8462f9\" (UID: \"0bc2b451-62aa-4cde-9af3-246f1f8462f9\") "
Jul 17 09:52:24.176035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:24.176007    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bc2b451-62aa-4cde-9af3-246f1f8462f9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0bc2b451-62aa-4cde-9af3-246f1f8462f9" (UID: "0bc2b451-62aa-4cde-9af3-246f1f8462f9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:52:24.276234 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:24.276191    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0bc2b451-62aa-4cde-9af3-246f1f8462f9-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:52:24.845196 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:24.845158    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"
Jul 17 09:52:24.845196 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:24.845172    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht" event={"ID":"0bc2b451-62aa-4cde-9af3-246f1f8462f9","Type":"ContainerDied","Data":"de81e67315a652dda2fbfa434c41e35857a3f1500887f877e723d69a35f45231"}
Jul 17 09:52:24.845429 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:24.845235    2571 scope.go:117] "RemoveContainer" containerID="0fca05632ccbc8900254396074e062180b581bd277d9fcfdbb34c126b63eaff4"
Jul 17 09:52:24.869753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:24.869725    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"]
Jul 17 09:52:24.876536 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:24.876512    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-pvjht"]
Jul 17 09:52:25.759546 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:25.759513    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" path="/var/lib/kubelet/pods/0bc2b451-62aa-4cde-9af3-246f1f8462f9/volumes"
Jul 17 09:52:32.752332 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:32.752301    2571 scope.go:117] "RemoveContainer" containerID="638bc6b1b824ac5b14436716916e9f3dd29b97061771a320b4b98392456cbaf0"
Jul 17 09:52:33.899413 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:33.899371    2571 generic.go:356] "Generic (PLEG): container finished" podID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerID="e338ac7867d160d46aa7a878ed937f2446775d00616a81a1aee57d00815b4867" exitCode=7
Jul 17 09:52:33.899904 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:33.899417    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm" event={"ID":"18d20ae0-0985-4d25-888e-6bdcdb9ac659","Type":"ContainerDied","Data":"e338ac7867d160d46aa7a878ed937f2446775d00616a81a1aee57d00815b4867"}
Jul 17 09:52:33.899904 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:33.899492    2571 scope.go:117] "RemoveContainer" containerID="638bc6b1b824ac5b14436716916e9f3dd29b97061771a320b4b98392456cbaf0"
Jul 17 09:52:33.899904 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:33.899733    2571 scope.go:117] "RemoveContainer" containerID="e338ac7867d160d46aa7a878ed937f2446775d00616a81a1aee57d00815b4867"
Jul 17 09:52:33.900049 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:52:33.899996    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm_odh-ai-gateway-infra(18d20ae0-0985-4d25-888e-6bdcdb9ac659)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659"
Jul 17 09:52:34.833476 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.833436    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"]
Jul 17 09:52:34.833938 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.833924    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerName="revoke-keys"
Jul 17 09:52:34.833994 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.833938    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerName="revoke-keys"
Jul 17 09:52:34.833994 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.833956    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerName="revoke-keys"
Jul 17 09:52:34.833994 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.833961    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerName="revoke-keys"
Jul 17 09:52:34.834095 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.834031    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerName="revoke-keys"
Jul 17 09:52:34.834095 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.834042    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerName="revoke-keys"
Jul 17 09:52:34.834095 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.834048    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerName="revoke-keys"
Jul 17 09:52:34.838391 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.838371    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"
Jul 17 09:52:34.854027 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.853993    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"]
Jul 17 09:52:34.970924 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.970887    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"]
Jul 17 09:52:34.975579 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:34.975548    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5575ca27-cba7-478d-8608-aa2adc4487e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x\" (UID: \"5575ca27-cba7-478d-8608-aa2adc4487e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"
Jul 17 09:52:35.077068 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.077028    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5575ca27-cba7-478d-8608-aa2adc4487e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x\" (UID: \"5575ca27-cba7-478d-8608-aa2adc4487e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"
Jul 17 09:52:35.077717 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.077692    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5575ca27-cba7-478d-8608-aa2adc4487e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x\" (UID: \"5575ca27-cba7-478d-8608-aa2adc4487e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"
Jul 17 09:52:35.116741 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.116712    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"
Jul 17 09:52:35.149083 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.149046    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"
Jul 17 09:52:35.279093 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.279055    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d20ae0-0985-4d25-888e-6bdcdb9ac659-maas-api-service-ca\") pod \"18d20ae0-0985-4d25-888e-6bdcdb9ac659\" (UID: \"18d20ae0-0985-4d25-888e-6bdcdb9ac659\") "
Jul 17 09:52:35.279406 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.279384    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18d20ae0-0985-4d25-888e-6bdcdb9ac659-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "18d20ae0-0985-4d25-888e-6bdcdb9ac659" (UID: "18d20ae0-0985-4d25-888e-6bdcdb9ac659"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:52:35.380915 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.380821    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18d20ae0-0985-4d25-888e-6bdcdb9ac659-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:52:35.514366 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.514343    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"]
Jul 17 09:52:35.516854 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:52:35.516825    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5575ca27_cba7_478d_8608_aa2adc4487e6.slice/crio-35cdda48051d445034bf001114c6e05421f00c6abd23646737eebbb811b376bb WatchSource:0}: Error finding container 35cdda48051d445034bf001114c6e05421f00c6abd23646737eebbb811b376bb: Status 404 returned error can't find the container with id 35cdda48051d445034bf001114c6e05421f00c6abd23646737eebbb811b376bb
Jul 17 09:52:35.925041 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.925002    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x" event={"ID":"5575ca27-cba7-478d-8608-aa2adc4487e6","Type":"ContainerStarted","Data":"30cc8f9bf35160ac2ab4b0f52930b12a4d111fc706c412ed316bc36836f18510"}
Jul 17 09:52:35.925230 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.925100    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x" event={"ID":"5575ca27-cba7-478d-8608-aa2adc4487e6","Type":"ContainerStarted","Data":"35cdda48051d445034bf001114c6e05421f00c6abd23646737eebbb811b376bb"}
Jul 17 09:52:35.926568 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.926541    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"
Jul 17 09:52:35.926684 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.926551    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm" event={"ID":"18d20ae0-0985-4d25-888e-6bdcdb9ac659","Type":"ContainerDied","Data":"e2698fb3b8af7f46e5620c438e3529059eea0ebb9c2d9e3b08ae93864bd797b6"}
Jul 17 09:52:35.926760 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.926708    2571 scope.go:117] "RemoveContainer" containerID="e338ac7867d160d46aa7a878ed937f2446775d00616a81a1aee57d00815b4867"
Jul 17 09:52:35.966646 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.966592    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x" podStartSLOduration=1.966575433 podStartE2EDuration="1.966575433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:52:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:52:35.965581511 +0000 UTC m=+2040.030402233" watchObservedRunningTime="2026-07-17 09:52:35.966575433 +0000 UTC m=+2040.031396155"
Jul 17 09:52:35.995872 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:35.995780    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"]
Jul 17 09:52:36.010735 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:36.010696    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lxdpm"]
Jul 17 09:52:36.933983 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:36.933946    2571 generic.go:356] "Generic (PLEG): container finished" podID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerID="30cc8f9bf35160ac2ab4b0f52930b12a4d111fc706c412ed316bc36836f18510" exitCode=6
Jul 17 09:52:36.934160 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:36.934030    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x" event={"ID":"5575ca27-cba7-478d-8608-aa2adc4487e6","Type":"ContainerDied","Data":"30cc8f9bf35160ac2ab4b0f52930b12a4d111fc706c412ed316bc36836f18510"}
Jul 17 09:52:36.934259 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:36.934243    2571 scope.go:117] "RemoveContainer" containerID="30cc8f9bf35160ac2ab4b0f52930b12a4d111fc706c412ed316bc36836f18510"
Jul 17 09:52:37.760109 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:37.760077    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" path="/var/lib/kubelet/pods/18d20ae0-0985-4d25-888e-6bdcdb9ac659/volumes"
Jul 17 09:52:37.942853 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:37.942818    2571 generic.go:356] "Generic (PLEG): container finished" podID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerID="7d502bb2d53b7fd2b908b382282a81141f1ee7fdee8e301e277047fcf9a1a38d" exitCode=6
Jul 17 09:52:37.943003 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:37.942900    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x" event={"ID":"5575ca27-cba7-478d-8608-aa2adc4487e6","Type":"ContainerDied","Data":"7d502bb2d53b7fd2b908b382282a81141f1ee7fdee8e301e277047fcf9a1a38d"}
Jul 17 09:52:37.943003 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:37.942961    2571 scope.go:117] "RemoveContainer" containerID="30cc8f9bf35160ac2ab4b0f52930b12a4d111fc706c412ed316bc36836f18510"
Jul 17 09:52:37.943188 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:37.943171    2571 scope.go:117] "RemoveContainer" containerID="7d502bb2d53b7fd2b908b382282a81141f1ee7fdee8e301e277047fcf9a1a38d"
Jul 17 09:52:37.943442 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:52:37.943418    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x_odh-ai-gateway-infra(5575ca27-cba7-478d-8608-aa2adc4487e6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6"
Jul 17 09:52:38.950810 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:38.950779    2571 scope.go:117] "RemoveContainer" containerID="7d502bb2d53b7fd2b908b382282a81141f1ee7fdee8e301e277047fcf9a1a38d"
Jul 17 09:52:38.951197 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:52:38.951048    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x_odh-ai-gateway-infra(5575ca27-cba7-478d-8608-aa2adc4487e6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6"
Jul 17 09:52:49.753123 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:49.753087    2571 scope.go:117] "RemoveContainer" containerID="7d502bb2d53b7fd2b908b382282a81141f1ee7fdee8e301e277047fcf9a1a38d"
Jul 17 09:52:51.023981 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:51.023947    2571 generic.go:356] "Generic (PLEG): container finished" podID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerID="5a25e6161e5dd724ef4c36ed98160e7f0b0516600434407a8369875d417e8442" exitCode=6
Jul 17 09:52:51.024386 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:51.024022    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x" event={"ID":"5575ca27-cba7-478d-8608-aa2adc4487e6","Type":"ContainerDied","Data":"5a25e6161e5dd724ef4c36ed98160e7f0b0516600434407a8369875d417e8442"}
Jul 17 09:52:51.024386 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:51.024082    2571 scope.go:117] "RemoveContainer" containerID="7d502bb2d53b7fd2b908b382282a81141f1ee7fdee8e301e277047fcf9a1a38d"
Jul 17 09:52:51.024509 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:51.024388    2571 scope.go:117] "RemoveContainer" containerID="5a25e6161e5dd724ef4c36ed98160e7f0b0516600434407a8369875d417e8442"
Jul 17 09:52:51.024649 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:52:51.024626    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x_odh-ai-gateway-infra(5575ca27-cba7-478d-8608-aa2adc4487e6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6"
Jul 17 09:52:52.141790 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:52.141751    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"]
Jul 17 09:52:52.295024 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:52.294993    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"
Jul 17 09:52:52.333149 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:52.333119    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5575ca27-cba7-478d-8608-aa2adc4487e6-maas-api-service-ca\") pod \"5575ca27-cba7-478d-8608-aa2adc4487e6\" (UID: \"5575ca27-cba7-478d-8608-aa2adc4487e6\") "
Jul 17 09:52:52.333503 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:52.333454    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5575ca27-cba7-478d-8608-aa2adc4487e6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5575ca27-cba7-478d-8608-aa2adc4487e6" (UID: "5575ca27-cba7-478d-8608-aa2adc4487e6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:52:52.434574 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:52.434453    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5575ca27-cba7-478d-8608-aa2adc4487e6-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:52:53.038543 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:53.038514    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"
Jul 17 09:52:53.038543 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:53.038524    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x" event={"ID":"5575ca27-cba7-478d-8608-aa2adc4487e6","Type":"ContainerDied","Data":"35cdda48051d445034bf001114c6e05421f00c6abd23646737eebbb811b376bb"}
Jul 17 09:52:53.038756 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:53.038585    2571 scope.go:117] "RemoveContainer" containerID="5a25e6161e5dd724ef4c36ed98160e7f0b0516600434407a8369875d417e8442"
Jul 17 09:52:53.080079 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:53.080038    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"]
Jul 17 09:52:53.094693 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:53.094666    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-7zj8x"]
Jul 17 09:52:53.759342 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:52:53.759299    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6" path="/var/lib/kubelet/pods/5575ca27-cba7-478d-8608-aa2adc4487e6/volumes"
Jul 17 09:53:04.565879 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.565793    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"]
Jul 17 09:53:04.566506 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566485    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerName="revoke-keys"
Jul 17 09:53:04.566565 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566511    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerName="revoke-keys"
Jul 17 09:53:04.566565 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566549    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerName="revoke-keys"
Jul 17 09:53:04.566565 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566558    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerName="revoke-keys"
Jul 17 09:53:04.566661 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566568    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerName="revoke-keys"
Jul 17 09:53:04.566661 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566577    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerName="revoke-keys"
Jul 17 09:53:04.566661 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566596    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerName="revoke-keys"
Jul 17 09:53:04.566661 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566605    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc2b451-62aa-4cde-9af3-246f1f8462f9" containerName="revoke-keys"
Jul 17 09:53:04.566661 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566622    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerName="revoke-keys"
Jul 17 09:53:04.566661 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566630    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerName="revoke-keys"
Jul 17 09:53:04.566843 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566724    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerName="revoke-keys"
Jul 17 09:53:04.566843 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566736    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerName="revoke-keys"
Jul 17 09:53:04.566843 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566744    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerName="revoke-keys"
Jul 17 09:53:04.566843 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566754    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerName="revoke-keys"
Jul 17 09:53:04.566843 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.566768    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerName="revoke-keys"
Jul 17 09:53:04.571753 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.571732    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"
Jul 17 09:53:04.574709 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.574688    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:53:04.595148 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.595117    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"]
Jul 17 09:53:04.643778 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.643731    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw\" (UID: \"f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"
Jul 17 09:53:04.745412 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.745372    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw\" (UID: \"f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"
Jul 17 09:53:04.746116 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.746088    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw\" (UID: \"f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"
Jul 17 09:53:04.882018 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:04.881913    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"
Jul 17 09:53:05.042020 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:05.041990    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"]
Jul 17 09:53:05.043875 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:53:05.043838    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf530bcd2_e37c_44d8_9bf4_ccca4f08bcfd.slice/crio-ebb948fdf167be3efdef103dd2ca379142f0836d29b6b4f51e5a4e9847238af8 WatchSource:0}: Error finding container ebb948fdf167be3efdef103dd2ca379142f0836d29b6b4f51e5a4e9847238af8: Status 404 returned error can't find the container with id ebb948fdf167be3efdef103dd2ca379142f0836d29b6b4f51e5a4e9847238af8
Jul 17 09:53:05.109529 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:05.109487    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw" event={"ID":"f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd","Type":"ContainerStarted","Data":"ebb948fdf167be3efdef103dd2ca379142f0836d29b6b4f51e5a4e9847238af8"}
Jul 17 09:53:06.116667 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:06.116630    2571 generic.go:356] "Generic (PLEG): container finished" podID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerID="25b60b7ea76da51c351abd2b5dfb66b29d72eb2cb5ec0a778adf2a0a48287774" exitCode=6
Jul 17 09:53:06.117071 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:06.116719    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw" event={"ID":"f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd","Type":"ContainerDied","Data":"25b60b7ea76da51c351abd2b5dfb66b29d72eb2cb5ec0a778adf2a0a48287774"}
Jul 17 09:53:06.117071 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:06.116881    2571 scope.go:117] "RemoveContainer" containerID="25b60b7ea76da51c351abd2b5dfb66b29d72eb2cb5ec0a778adf2a0a48287774"
Jul 17 09:53:07.123977 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:07.123889    2571 generic.go:356] "Generic (PLEG): container finished" podID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerID="9cd6daf056b75ffa7ef9dbdf6f1f67a4eccc80ca55b63bde45a9430ec0cc7ad0" exitCode=6
Jul 17 09:53:07.124362 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:07.123974    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw" event={"ID":"f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd","Type":"ContainerDied","Data":"9cd6daf056b75ffa7ef9dbdf6f1f67a4eccc80ca55b63bde45a9430ec0cc7ad0"}
Jul 17 09:53:07.124362 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:07.124047    2571 scope.go:117] "RemoveContainer" containerID="25b60b7ea76da51c351abd2b5dfb66b29d72eb2cb5ec0a778adf2a0a48287774"
Jul 17 09:53:07.124362 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:07.124240    2571 scope.go:117] "RemoveContainer" containerID="9cd6daf056b75ffa7ef9dbdf6f1f67a4eccc80ca55b63bde45a9430ec0cc7ad0"
Jul 17 09:53:07.124559 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:53:07.124519    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw_odh-ai-gateway-infra(f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd"
Jul 17 09:53:08.131704 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:08.131675    2571 scope.go:117] "RemoveContainer" containerID="9cd6daf056b75ffa7ef9dbdf6f1f67a4eccc80ca55b63bde45a9430ec0cc7ad0"
Jul 17 09:53:08.132101 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:53:08.131903    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw_odh-ai-gateway-infra(f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd"
Jul 17 09:53:13.563058 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.563023    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"]
Jul 17 09:53:13.563452 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.563439    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerName="revoke-keys"
Jul 17 09:53:13.563509 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.563453    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d20ae0-0985-4d25-888e-6bdcdb9ac659" containerName="revoke-keys"
Jul 17 09:53:13.563509 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.563498    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerName="revoke-keys"
Jul 17 09:53:13.563509 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.563504    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerName="revoke-keys"
Jul 17 09:53:13.563611 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.563580    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5575ca27-cba7-478d-8608-aa2adc4487e6" containerName="revoke-keys"
Jul 17 09:53:13.566758 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.566742    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"
Jul 17 09:53:13.576157 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.576131    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"]
Jul 17 09:53:13.730006 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.729953    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2319d89c-6bc0-4003-8aad-1b1dc648d247-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt\" (UID: \"2319d89c-6bc0-4003-8aad-1b1dc648d247\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"
Jul 17 09:53:13.830881 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.830783    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2319d89c-6bc0-4003-8aad-1b1dc648d247-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt\" (UID: \"2319d89c-6bc0-4003-8aad-1b1dc648d247\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"
Jul 17 09:53:13.831558 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.831531    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2319d89c-6bc0-4003-8aad-1b1dc648d247-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt\" (UID: \"2319d89c-6bc0-4003-8aad-1b1dc648d247\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"
Jul 17 09:53:13.879331 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:13.879296    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"
Jul 17 09:53:14.022319 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:14.022280    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"]
Jul 17 09:53:14.023867 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:53:14.023835    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2319d89c_6bc0_4003_8aad_1b1dc648d247.slice/crio-781b279506b967d5e75cba74ec174db0cc955ba9e8163ef56163cfa7dc2b1d3b WatchSource:0}: Error finding container 781b279506b967d5e75cba74ec174db0cc955ba9e8163ef56163cfa7dc2b1d3b: Status 404 returned error can't find the container with id 781b279506b967d5e75cba74ec174db0cc955ba9e8163ef56163cfa7dc2b1d3b
Jul 17 09:53:14.168198 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:14.168117    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt" event={"ID":"2319d89c-6bc0-4003-8aad-1b1dc648d247","Type":"ContainerStarted","Data":"781b279506b967d5e75cba74ec174db0cc955ba9e8163ef56163cfa7dc2b1d3b"}
Jul 17 09:53:15.175917 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:15.175869    2571 generic.go:356] "Generic (PLEG): container finished" podID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerID="82abaaecc1f87ce19d4e9dfca252a7de674ac2a9d4aef4e8830272c5b7a1f8a4" exitCode=7
Jul 17 09:53:15.176380 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:15.175958    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt" event={"ID":"2319d89c-6bc0-4003-8aad-1b1dc648d247","Type":"ContainerDied","Data":"82abaaecc1f87ce19d4e9dfca252a7de674ac2a9d4aef4e8830272c5b7a1f8a4"}
Jul 17 09:53:15.176380 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:15.176179    2571 scope.go:117] "RemoveContainer" containerID="82abaaecc1f87ce19d4e9dfca252a7de674ac2a9d4aef4e8830272c5b7a1f8a4"
Jul 17 09:53:16.183763 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:16.183670    2571 generic.go:356] "Generic (PLEG): container finished" podID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerID="b23c746694d31bf4228d16e96593f0f1be2aad100edab546ea2365e445111bce" exitCode=7
Jul 17 09:53:16.184193 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:16.183754    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt" event={"ID":"2319d89c-6bc0-4003-8aad-1b1dc648d247","Type":"ContainerDied","Data":"b23c746694d31bf4228d16e96593f0f1be2aad100edab546ea2365e445111bce"}
Jul 17 09:53:16.184193 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:16.183845    2571 scope.go:117] "RemoveContainer" containerID="82abaaecc1f87ce19d4e9dfca252a7de674ac2a9d4aef4e8830272c5b7a1f8a4"
Jul 17 09:53:16.184193 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:16.184065    2571 scope.go:117] "RemoveContainer" containerID="b23c746694d31bf4228d16e96593f0f1be2aad100edab546ea2365e445111bce"
Jul 17 09:53:16.184363 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:53:16.184307    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt_odh-ai-gateway-infra(2319d89c-6bc0-4003-8aad-1b1dc648d247)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247"
Jul 17 09:53:17.191130 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:17.191102    2571 scope.go:117] "RemoveContainer" containerID="b23c746694d31bf4228d16e96593f0f1be2aad100edab546ea2365e445111bce"
Jul 17 09:53:17.191510 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:53:17.191352    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt_odh-ai-gateway-infra(2319d89c-6bc0-4003-8aad-1b1dc648d247)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247"
Jul 17 09:53:21.752387 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:21.752353    2571 scope.go:117] "RemoveContainer" containerID="9cd6daf056b75ffa7ef9dbdf6f1f67a4eccc80ca55b63bde45a9430ec0cc7ad0"
Jul 17 09:53:22.221020 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:22.220980    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw" event={"ID":"f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd","Type":"ContainerStarted","Data":"7e85c3ab5a88f8be39525ef6f7f33ebf42839cef64835014ef5d624e65ab9976"}
Jul 17 09:53:22.251408 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:22.251346    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw" podStartSLOduration=18.251332171 podStartE2EDuration="18.251332171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:53:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:53:22.249688559 +0000 UTC m=+2086.314509281" watchObservedRunningTime="2026-07-17 09:53:22.251332171 +0000 UTC m=+2086.316152894"
Jul 17 09:53:23.229078 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:23.229044    2571 generic.go:356] "Generic (PLEG): container finished" podID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerID="7e85c3ab5a88f8be39525ef6f7f33ebf42839cef64835014ef5d624e65ab9976" exitCode=6
Jul 17 09:53:23.229545 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:23.229120    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw" event={"ID":"f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd","Type":"ContainerDied","Data":"7e85c3ab5a88f8be39525ef6f7f33ebf42839cef64835014ef5d624e65ab9976"}
Jul 17 09:53:23.229545 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:23.229186    2571 scope.go:117] "RemoveContainer" containerID="9cd6daf056b75ffa7ef9dbdf6f1f67a4eccc80ca55b63bde45a9430ec0cc7ad0"
Jul 17 09:53:23.229545 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:23.229524    2571 scope.go:117] "RemoveContainer" containerID="7e85c3ab5a88f8be39525ef6f7f33ebf42839cef64835014ef5d624e65ab9976"
Jul 17 09:53:23.229806 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:53:23.229780    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw_odh-ai-gateway-infra(f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd"
Jul 17 09:53:23.272330 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:23.272297    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"]
Jul 17 09:53:24.386524 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.386490    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"
Jul 17 09:53:24.537726 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.537627    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd-maas-api-service-ca\") pod \"f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd\" (UID: \"f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd\") "
Jul 17 09:53:24.538035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.538011    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" (UID: "f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:53:24.638387 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.638348    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:53:24.882597 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.882511    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"]
Jul 17 09:53:24.882992 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.882980    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerName="revoke-keys"
Jul 17 09:53:24.883035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.882994    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerName="revoke-keys"
Jul 17 09:53:24.883035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.883014    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerName="revoke-keys"
Jul 17 09:53:24.883035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.883020    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerName="revoke-keys"
Jul 17 09:53:24.883035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.883026    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerName="revoke-keys"
Jul 17 09:53:24.883035 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.883031    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerName="revoke-keys"
Jul 17 09:53:24.883194 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.883109    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerName="revoke-keys"
Jul 17 09:53:24.883194 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.883122    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerName="revoke-keys"
Jul 17 09:53:24.886272 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.886252    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"
Jul 17 09:53:24.898850 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.898817    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"]
Jul 17 09:53:24.940418 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:24.940377    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a181b46-67f5-4d4e-bf00-742145bfe006-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk\" (UID: \"7a181b46-67f5-4d4e-bf00-742145bfe006\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"
Jul 17 09:53:25.040818 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:25.040776    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a181b46-67f5-4d4e-bf00-742145bfe006-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk\" (UID: \"7a181b46-67f5-4d4e-bf00-742145bfe006\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"
Jul 17 09:53:25.041441 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:25.041419    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a181b46-67f5-4d4e-bf00-742145bfe006-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk\" (UID: \"7a181b46-67f5-4d4e-bf00-742145bfe006\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"
Jul 17 09:53:25.197676 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:25.197636    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"
Jul 17 09:53:25.246267 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:25.246230    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"
Jul 17 09:53:25.246267 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:25.246251    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw" event={"ID":"f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd","Type":"ContainerDied","Data":"ebb948fdf167be3efdef103dd2ca379142f0836d29b6b4f51e5a4e9847238af8"}
Jul 17 09:53:25.246503 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:25.246350    2571 scope.go:117] "RemoveContainer" containerID="7e85c3ab5a88f8be39525ef6f7f33ebf42839cef64835014ef5d624e65ab9976"
Jul 17 09:53:25.281818 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:25.281768    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"]
Jul 17 09:53:25.285036 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:25.285004    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mtbqw"]
Jul 17 09:53:25.360887 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:25.360850    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"]
Jul 17 09:53:25.362924 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:53:25.362893    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a181b46_67f5_4d4e_bf00_742145bfe006.slice/crio-14e3f483b5ede840206b08156f906c508b8f418b16f90527df1d0f7fda1e95c4 WatchSource:0}: Error finding container 14e3f483b5ede840206b08156f906c508b8f418b16f90527df1d0f7fda1e95c4: Status 404 returned error can't find the container with id 14e3f483b5ede840206b08156f906c508b8f418b16f90527df1d0f7fda1e95c4
Jul 17 09:53:25.759714 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:25.759668    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" path="/var/lib/kubelet/pods/f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd/volumes"
Jul 17 09:53:26.253863 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:26.253827    2571 generic.go:356] "Generic (PLEG): container finished" podID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerID="738cdec22fd519c7703f819019713398ffb883c0396f6ad14dc8cc321b4060fe" exitCode=6
Jul 17 09:53:26.254072 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:26.253917    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk" event={"ID":"7a181b46-67f5-4d4e-bf00-742145bfe006","Type":"ContainerDied","Data":"738cdec22fd519c7703f819019713398ffb883c0396f6ad14dc8cc321b4060fe"}
Jul 17 09:53:26.254072 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:26.253982    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk" event={"ID":"7a181b46-67f5-4d4e-bf00-742145bfe006","Type":"ContainerStarted","Data":"14e3f483b5ede840206b08156f906c508b8f418b16f90527df1d0f7fda1e95c4"}
Jul 17 09:53:26.254193 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:26.254112    2571 scope.go:117] "RemoveContainer" containerID="738cdec22fd519c7703f819019713398ffb883c0396f6ad14dc8cc321b4060fe"
Jul 17 09:53:27.263525 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:27.263488    2571 generic.go:356] "Generic (PLEG): container finished" podID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerID="df66c7854f2fa9b686e1e35eec87794fcc6d8a6db5e65f73288953fbc4d47160" exitCode=6
Jul 17 09:53:27.264010 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:27.263577    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk" event={"ID":"7a181b46-67f5-4d4e-bf00-742145bfe006","Type":"ContainerDied","Data":"df66c7854f2fa9b686e1e35eec87794fcc6d8a6db5e65f73288953fbc4d47160"}
Jul 17 09:53:27.264010 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:27.263630    2571 scope.go:117] "RemoveContainer" containerID="738cdec22fd519c7703f819019713398ffb883c0396f6ad14dc8cc321b4060fe"
Jul 17 09:53:27.264010 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:27.263892    2571 scope.go:117] "RemoveContainer" containerID="df66c7854f2fa9b686e1e35eec87794fcc6d8a6db5e65f73288953fbc4d47160"
Jul 17 09:53:27.264178 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:53:27.264140    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk_odh-ai-gateway-infra(7a181b46-67f5-4d4e-bf00-742145bfe006)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006"
Jul 17 09:53:28.270377 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:28.270347    2571 scope.go:117] "RemoveContainer" containerID="df66c7854f2fa9b686e1e35eec87794fcc6d8a6db5e65f73288953fbc4d47160"
Jul 17 09:53:28.270774 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:53:28.270598    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk_odh-ai-gateway-infra(7a181b46-67f5-4d4e-bf00-742145bfe006)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006"
Jul 17 09:53:31.752155 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:31.752118    2571 scope.go:117] "RemoveContainer" containerID="b23c746694d31bf4228d16e96593f0f1be2aad100edab546ea2365e445111bce"
Jul 17 09:53:32.295521 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:32.295402    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt" event={"ID":"2319d89c-6bc0-4003-8aad-1b1dc648d247","Type":"ContainerStarted","Data":"6305b15072a7c88335058145940133e4097b58aacaa1ea5742fe68d0c48e8910"}
Jul 17 09:53:32.316482 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:32.316408    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt" podStartSLOduration=19.316391066 podStartE2EDuration="19.316391066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:53:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:53:32.315707873 +0000 UTC m=+2096.380528591" watchObservedRunningTime="2026-07-17 09:53:32.316391066 +0000 UTC m=+2096.381211788"
Jul 17 09:53:33.303217 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:33.303183    2571 generic.go:356] "Generic (PLEG): container finished" podID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerID="6305b15072a7c88335058145940133e4097b58aacaa1ea5742fe68d0c48e8910" exitCode=7
Jul 17 09:53:33.303629 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:33.303255    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt" event={"ID":"2319d89c-6bc0-4003-8aad-1b1dc648d247","Type":"ContainerDied","Data":"6305b15072a7c88335058145940133e4097b58aacaa1ea5742fe68d0c48e8910"}
Jul 17 09:53:33.303629 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:33.303332    2571 scope.go:117] "RemoveContainer" containerID="b23c746694d31bf4228d16e96593f0f1be2aad100edab546ea2365e445111bce"
Jul 17 09:53:33.303629 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:33.303580    2571 scope.go:117] "RemoveContainer" containerID="6305b15072a7c88335058145940133e4097b58aacaa1ea5742fe68d0c48e8910"
Jul 17 09:53:33.303852 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:53:33.303833    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt_odh-ai-gateway-infra(2319d89c-6bc0-4003-8aad-1b1dc648d247)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247"
Jul 17 09:53:33.343851 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:33.343790    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"]
Jul 17 09:53:34.467719 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:34.467685    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"
Jul 17 09:53:34.536724 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:34.536689    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2319d89c-6bc0-4003-8aad-1b1dc648d247-maas-api-service-ca\") pod \"2319d89c-6bc0-4003-8aad-1b1dc648d247\" (UID: \"2319d89c-6bc0-4003-8aad-1b1dc648d247\") "
Jul 17 09:53:34.537092 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:34.537065    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2319d89c-6bc0-4003-8aad-1b1dc648d247-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2319d89c-6bc0-4003-8aad-1b1dc648d247" (UID: "2319d89c-6bc0-4003-8aad-1b1dc648d247"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:53:34.637408 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:34.637311    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2319d89c-6bc0-4003-8aad-1b1dc648d247-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:53:35.317923 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:35.317892    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"
Jul 17 09:53:35.318157 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:35.317890    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt" event={"ID":"2319d89c-6bc0-4003-8aad-1b1dc648d247","Type":"ContainerDied","Data":"781b279506b967d5e75cba74ec174db0cc955ba9e8163ef56163cfa7dc2b1d3b"}
Jul 17 09:53:35.318157 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:35.318052    2571 scope.go:117] "RemoveContainer" containerID="6305b15072a7c88335058145940133e4097b58aacaa1ea5742fe68d0c48e8910"
Jul 17 09:53:35.344656 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:35.344622    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"]
Jul 17 09:53:35.350580 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:35.350549    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-b8xrt"]
Jul 17 09:53:35.759122 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:35.759084    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247" path="/var/lib/kubelet/pods/2319d89c-6bc0-4003-8aad-1b1dc648d247/volumes"
Jul 17 09:53:37.512785 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:37.512755    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:53:37.520854 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:37.520830    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:53:42.752556 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:42.752522    2571 scope.go:117] "RemoveContainer" containerID="df66c7854f2fa9b686e1e35eec87794fcc6d8a6db5e65f73288953fbc4d47160"
Jul 17 09:53:43.367440 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:43.367401    2571 generic.go:356] "Generic (PLEG): container finished" podID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerID="a8b04c906bbc869828cb81a0d47e542b2b6f19605fc96d8be6ddad2bad66deca" exitCode=6
Jul 17 09:53:43.367601 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:43.367447    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk" event={"ID":"7a181b46-67f5-4d4e-bf00-742145bfe006","Type":"ContainerDied","Data":"a8b04c906bbc869828cb81a0d47e542b2b6f19605fc96d8be6ddad2bad66deca"}
Jul 17 09:53:43.367601 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:43.367543    2571 scope.go:117] "RemoveContainer" containerID="df66c7854f2fa9b686e1e35eec87794fcc6d8a6db5e65f73288953fbc4d47160"
Jul 17 09:53:43.367939 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:43.367916    2571 scope.go:117] "RemoveContainer" containerID="a8b04c906bbc869828cb81a0d47e542b2b6f19605fc96d8be6ddad2bad66deca"
Jul 17 09:53:43.368244 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:53:43.368211    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk_odh-ai-gateway-infra(7a181b46-67f5-4d4e-bf00-742145bfe006)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006"
Jul 17 09:53:44.399328 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:44.399288    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"]
Jul 17 09:53:44.545742 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:44.545713    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"
Jul 17 09:53:44.630673 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:44.630635    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a181b46-67f5-4d4e-bf00-742145bfe006-maas-api-service-ca\") pod \"7a181b46-67f5-4d4e-bf00-742145bfe006\" (UID: \"7a181b46-67f5-4d4e-bf00-742145bfe006\") "
Jul 17 09:53:44.631004 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:44.630980    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a181b46-67f5-4d4e-bf00-742145bfe006-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7a181b46-67f5-4d4e-bf00-742145bfe006" (UID: "7a181b46-67f5-4d4e-bf00-742145bfe006"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:53:44.732048 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:44.732007    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a181b46-67f5-4d4e-bf00-742145bfe006-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:53:45.382166 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:45.382128    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk" event={"ID":"7a181b46-67f5-4d4e-bf00-742145bfe006","Type":"ContainerDied","Data":"14e3f483b5ede840206b08156f906c508b8f418b16f90527df1d0f7fda1e95c4"}
Jul 17 09:53:45.382166 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:45.382156    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"
Jul 17 09:53:45.382410 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:45.382200    2571 scope.go:117] "RemoveContainer" containerID="a8b04c906bbc869828cb81a0d47e542b2b6f19605fc96d8be6ddad2bad66deca"
Jul 17 09:53:45.406227 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:45.406195    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"]
Jul 17 09:53:45.411310 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:45.411283    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-nzpdk"]
Jul 17 09:53:45.758405 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:53:45.758358    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006" path="/var/lib/kubelet/pods/7a181b46-67f5-4d4e-bf00-742145bfe006/volumes"
Jul 17 09:54:04.598543 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.598499    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"]
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.598920    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerName="revoke-keys"
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.598932    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerName="revoke-keys"
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.598951    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerName="revoke-keys"
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.598956    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerName="revoke-keys"
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.598964    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerName="revoke-keys"
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.598970    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerName="revoke-keys"
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.598976    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerName="revoke-keys"
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.598982    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerName="revoke-keys"
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.598994    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerName="revoke-keys"
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.598999    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerName="revoke-keys"
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.599005    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerName="revoke-keys"
Jul 17 09:54:04.599005 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.599011    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerName="revoke-keys"
Jul 17 09:54:04.599394 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.599088    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerName="revoke-keys"
Jul 17 09:54:04.599394 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.599096    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerName="revoke-keys"
Jul 17 09:54:04.599394 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.599103    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerName="revoke-keys"
Jul 17 09:54:04.599394 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.599108    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerName="revoke-keys"
Jul 17 09:54:04.599394 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.599117    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f530bcd2-e37c-44d8-9bf4-ccca4f08bcfd" containerName="revoke-keys"
Jul 17 09:54:04.603620 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.603601    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"
Jul 17 09:54:04.606575 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.606548    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:54:04.611646 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.611057    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"]
Jul 17 09:54:04.719896 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.719858    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82f43c45-c706-4f66-a0c3-5b4b11eca5a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf\" (UID: \"82f43c45-c706-4f66-a0c3-5b4b11eca5a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"
Jul 17 09:54:04.821083 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.821034    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82f43c45-c706-4f66-a0c3-5b4b11eca5a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf\" (UID: \"82f43c45-c706-4f66-a0c3-5b4b11eca5a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"
Jul 17 09:54:04.821759 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.821740    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82f43c45-c706-4f66-a0c3-5b4b11eca5a5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf\" (UID: \"82f43c45-c706-4f66-a0c3-5b4b11eca5a5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"
Jul 17 09:54:04.915388 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:04.915274    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"
Jul 17 09:54:05.062506 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:05.062455    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"]
Jul 17 09:54:05.063778 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:54:05.063750    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f43c45_c706_4f66_a0c3_5b4b11eca5a5.slice/crio-08f0380b7324a091f28dd183463a6a6801b79c01122bf5b149d9d78f8f0c16ed WatchSource:0}: Error finding container 08f0380b7324a091f28dd183463a6a6801b79c01122bf5b149d9d78f8f0c16ed: Status 404 returned error can't find the container with id 08f0380b7324a091f28dd183463a6a6801b79c01122bf5b149d9d78f8f0c16ed
Jul 17 09:54:05.502530 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:05.502406    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf" event={"ID":"82f43c45-c706-4f66-a0c3-5b4b11eca5a5","Type":"ContainerStarted","Data":"8d1398847e1e9a521164595fba91efe75b5267b05dd825b6dbb55b98e6e4c1e2"}
Jul 17 09:54:05.502530 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:05.502489    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf" event={"ID":"82f43c45-c706-4f66-a0c3-5b4b11eca5a5","Type":"ContainerStarted","Data":"08f0380b7324a091f28dd183463a6a6801b79c01122bf5b149d9d78f8f0c16ed"}
Jul 17 09:54:05.523331 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:05.523269    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf" podStartSLOduration=1.523250803 podStartE2EDuration="1.523250803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:54:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:54:05.520863025 +0000 UTC m=+2129.585683747" watchObservedRunningTime="2026-07-17 09:54:05.523250803 +0000 UTC m=+2129.588071525"
Jul 17 09:54:06.509802 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:06.509765    2571 generic.go:356] "Generic (PLEG): container finished" podID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerID="8d1398847e1e9a521164595fba91efe75b5267b05dd825b6dbb55b98e6e4c1e2" exitCode=6
Jul 17 09:54:06.510211 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:06.509803    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf" event={"ID":"82f43c45-c706-4f66-a0c3-5b4b11eca5a5","Type":"ContainerDied","Data":"8d1398847e1e9a521164595fba91efe75b5267b05dd825b6dbb55b98e6e4c1e2"}
Jul 17 09:54:06.510211 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:06.510092    2571 scope.go:117] "RemoveContainer" containerID="8d1398847e1e9a521164595fba91efe75b5267b05dd825b6dbb55b98e6e4c1e2"
Jul 17 09:54:07.518103 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:07.518012    2571 generic.go:356] "Generic (PLEG): container finished" podID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerID="a0e8cfdafd386896ea6eb655d253d25c0b8bc4b5e91cd4eaa6b2b442c141f266" exitCode=6
Jul 17 09:54:07.518584 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:07.518102    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf" event={"ID":"82f43c45-c706-4f66-a0c3-5b4b11eca5a5","Type":"ContainerDied","Data":"a0e8cfdafd386896ea6eb655d253d25c0b8bc4b5e91cd4eaa6b2b442c141f266"}
Jul 17 09:54:07.518584 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:07.518167    2571 scope.go:117] "RemoveContainer" containerID="8d1398847e1e9a521164595fba91efe75b5267b05dd825b6dbb55b98e6e4c1e2"
Jul 17 09:54:07.518584 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:07.518348    2571 scope.go:117] "RemoveContainer" containerID="a0e8cfdafd386896ea6eb655d253d25c0b8bc4b5e91cd4eaa6b2b442c141f266"
Jul 17 09:54:07.518738 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:54:07.518593    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf_odh-ai-gateway-infra(82f43c45-c706-4f66-a0c3-5b4b11eca5a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5"
Jul 17 09:54:08.526534 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:08.526498    2571 scope.go:117] "RemoveContainer" containerID="a0e8cfdafd386896ea6eb655d253d25c0b8bc4b5e91cd4eaa6b2b442c141f266"
Jul 17 09:54:08.526929 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:54:08.526697    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf_odh-ai-gateway-infra(82f43c45-c706-4f66-a0c3-5b4b11eca5a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5"
Jul 17 09:54:13.611221 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:13.611174    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"]
Jul 17 09:54:13.611796 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:13.611765    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a181b46-67f5-4d4e-bf00-742145bfe006" containerName="revoke-keys"
Jul 17 09:54:13.611844 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:13.611801    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2319d89c-6bc0-4003-8aad-1b1dc648d247" containerName="revoke-keys"
Jul 17 09:54:13.614916 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:13.614900    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"
Jul 17 09:54:13.626730 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:13.626700    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"]
Jul 17 09:54:13.697391 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:13.697343    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a2d033-6177-41d7-adc0-19e292c20d9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5\" (UID: \"a6a2d033-6177-41d7-adc0-19e292c20d9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"
Jul 17 09:54:13.798326 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:13.798284    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a2d033-6177-41d7-adc0-19e292c20d9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5\" (UID: \"a6a2d033-6177-41d7-adc0-19e292c20d9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"
Jul 17 09:54:13.799028 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:13.799002    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a2d033-6177-41d7-adc0-19e292c20d9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5\" (UID: \"a6a2d033-6177-41d7-adc0-19e292c20d9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"
Jul 17 09:54:13.926246 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:13.926144    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"
Jul 17 09:54:14.073725 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:14.073694    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"]
Jul 17 09:54:14.075187 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:54:14.075158    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a2d033_6177_41d7_adc0_19e292c20d9e.slice/crio-692c669843112c0ac3905ab25ae47d6b92e236a6aa0d3178fed68718f3685048 WatchSource:0}: Error finding container 692c669843112c0ac3905ab25ae47d6b92e236a6aa0d3178fed68718f3685048: Status 404 returned error can't find the container with id 692c669843112c0ac3905ab25ae47d6b92e236a6aa0d3178fed68718f3685048
Jul 17 09:54:14.562258 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:14.562158    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5" event={"ID":"a6a2d033-6177-41d7-adc0-19e292c20d9e","Type":"ContainerStarted","Data":"378fe849b6f7785df93ee65768ecacdae152a49d4ebbc7c20beed3b3cfd6bdec"}
Jul 17 09:54:14.562258 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:14.562220    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5" event={"ID":"a6a2d033-6177-41d7-adc0-19e292c20d9e","Type":"ContainerStarted","Data":"692c669843112c0ac3905ab25ae47d6b92e236a6aa0d3178fed68718f3685048"}
Jul 17 09:54:14.581302 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:14.581250    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5" podStartSLOduration=1.5812325139999999 podStartE2EDuration="1.581232514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:54:14.579375266 +0000 UTC m=+2138.644195992" watchObservedRunningTime="2026-07-17 09:54:14.581232514 +0000 UTC m=+2138.646053236"
Jul 17 09:54:15.569701 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:15.569662    2571 generic.go:356] "Generic (PLEG): container finished" podID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerID="378fe849b6f7785df93ee65768ecacdae152a49d4ebbc7c20beed3b3cfd6bdec" exitCode=7
Jul 17 09:54:15.570111 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:15.569745    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5" event={"ID":"a6a2d033-6177-41d7-adc0-19e292c20d9e","Type":"ContainerDied","Data":"378fe849b6f7785df93ee65768ecacdae152a49d4ebbc7c20beed3b3cfd6bdec"}
Jul 17 09:54:15.570111 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:15.569988    2571 scope.go:117] "RemoveContainer" containerID="378fe849b6f7785df93ee65768ecacdae152a49d4ebbc7c20beed3b3cfd6bdec"
Jul 17 09:54:16.576770 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:16.576731    2571 generic.go:356] "Generic (PLEG): container finished" podID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerID="c41adc71b076b48cf908f2f1d1f00165536017b474d8603d895b4508955153d2" exitCode=7
Jul 17 09:54:16.577185 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:16.576784    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5" event={"ID":"a6a2d033-6177-41d7-adc0-19e292c20d9e","Type":"ContainerDied","Data":"c41adc71b076b48cf908f2f1d1f00165536017b474d8603d895b4508955153d2"}
Jul 17 09:54:16.577185 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:16.576846    2571 scope.go:117] "RemoveContainer" containerID="378fe849b6f7785df93ee65768ecacdae152a49d4ebbc7c20beed3b3cfd6bdec"
Jul 17 09:54:16.577185 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:16.577056    2571 scope.go:117] "RemoveContainer" containerID="c41adc71b076b48cf908f2f1d1f00165536017b474d8603d895b4508955153d2"
Jul 17 09:54:16.577311 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:54:16.577278    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5_odh-ai-gateway-infra(a6a2d033-6177-41d7-adc0-19e292c20d9e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e"
Jul 17 09:54:17.585121 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:17.585092    2571 scope.go:117] "RemoveContainer" containerID="c41adc71b076b48cf908f2f1d1f00165536017b474d8603d895b4508955153d2"
Jul 17 09:54:17.585627 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:54:17.585357    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5_odh-ai-gateway-infra(a6a2d033-6177-41d7-adc0-19e292c20d9e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e"
Jul 17 09:54:18.752845 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:18.752810    2571 scope.go:117] "RemoveContainer" containerID="a0e8cfdafd386896ea6eb655d253d25c0b8bc4b5e91cd4eaa6b2b442c141f266"
Jul 17 09:54:19.597419 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:19.597381    2571 generic.go:356] "Generic (PLEG): container finished" podID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerID="e2f048183b037d35aa59101aaf02839aa300bdef0051b582a5b25df4a4fd2a23" exitCode=6
Jul 17 09:54:19.597625 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:19.597429    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf" event={"ID":"82f43c45-c706-4f66-a0c3-5b4b11eca5a5","Type":"ContainerDied","Data":"e2f048183b037d35aa59101aaf02839aa300bdef0051b582a5b25df4a4fd2a23"}
Jul 17 09:54:19.597625 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:19.597534    2571 scope.go:117] "RemoveContainer" containerID="a0e8cfdafd386896ea6eb655d253d25c0b8bc4b5e91cd4eaa6b2b442c141f266"
Jul 17 09:54:19.597909 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:19.597879    2571 scope.go:117] "RemoveContainer" containerID="e2f048183b037d35aa59101aaf02839aa300bdef0051b582a5b25df4a4fd2a23"
Jul 17 09:54:19.598170 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:54:19.598144    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf_odh-ai-gateway-infra(82f43c45-c706-4f66-a0c3-5b4b11eca5a5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5"
Jul 17 09:54:20.629892 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:20.629856    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"]
Jul 17 09:54:20.775238 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:20.775211    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"
Jul 17 09:54:20.855966 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:20.855925    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82f43c45-c706-4f66-a0c3-5b4b11eca5a5-maas-api-service-ca\") pod \"82f43c45-c706-4f66-a0c3-5b4b11eca5a5\" (UID: \"82f43c45-c706-4f66-a0c3-5b4b11eca5a5\") "
Jul 17 09:54:20.856303 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:20.856281    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82f43c45-c706-4f66-a0c3-5b4b11eca5a5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "82f43c45-c706-4f66-a0c3-5b4b11eca5a5" (UID: "82f43c45-c706-4f66-a0c3-5b4b11eca5a5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:54:20.957097 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:20.957064    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82f43c45-c706-4f66-a0c3-5b4b11eca5a5-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:54:21.612247 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:21.612217    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"
Jul 17 09:54:21.612427 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:21.612216    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf" event={"ID":"82f43c45-c706-4f66-a0c3-5b4b11eca5a5","Type":"ContainerDied","Data":"08f0380b7324a091f28dd183463a6a6801b79c01122bf5b149d9d78f8f0c16ed"}
Jul 17 09:54:21.612427 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:21.612348    2571 scope.go:117] "RemoveContainer" containerID="e2f048183b037d35aa59101aaf02839aa300bdef0051b582a5b25df4a4fd2a23"
Jul 17 09:54:21.638643 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:21.638601    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"]
Jul 17 09:54:21.642552 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:21.642523    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-s8phf"]
Jul 17 09:54:21.758477 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:21.758430    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" path="/var/lib/kubelet/pods/82f43c45-c706-4f66-a0c3-5b4b11eca5a5/volumes"
Jul 17 09:54:24.940162 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:24.940125    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"]
Jul 17 09:54:24.940609 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:24.940595    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerName="revoke-keys"
Jul 17 09:54:24.940669 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:24.940613    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerName="revoke-keys"
Jul 17 09:54:24.940669 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:24.940633    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerName="revoke-keys"
Jul 17 09:54:24.940669 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:24.940639    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerName="revoke-keys"
Jul 17 09:54:24.940762 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:24.940724    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerName="revoke-keys"
Jul 17 09:54:24.940762 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:24.940733    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerName="revoke-keys"
Jul 17 09:54:24.940762 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:24.940740    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerName="revoke-keys"
Jul 17 09:54:24.945335 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:24.945307    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"
Jul 17 09:54:24.952608 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:24.952575    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"]
Jul 17 09:54:24.992845 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:24.992808    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81815632-0afd-4111-bf76-808daa743644-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp\" (UID: \"81815632-0afd-4111-bf76-808daa743644\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"
Jul 17 09:54:25.093444 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:25.093394    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81815632-0afd-4111-bf76-808daa743644-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp\" (UID: \"81815632-0afd-4111-bf76-808daa743644\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"
Jul 17 09:54:25.094101 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:25.094078    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81815632-0afd-4111-bf76-808daa743644-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp\" (UID: \"81815632-0afd-4111-bf76-808daa743644\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"
Jul 17 09:54:25.261890 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:25.261801    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"
Jul 17 09:54:25.416175 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:25.416146    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"]
Jul 17 09:54:25.418205 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:54:25.418176    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81815632_0afd_4111_bf76_808daa743644.slice/crio-03e93882a4dfc74c46252279aa656c038d38a09b28940768a2abc2d02e918aad WatchSource:0}: Error finding container 03e93882a4dfc74c46252279aa656c038d38a09b28940768a2abc2d02e918aad: Status 404 returned error can't find the container with id 03e93882a4dfc74c46252279aa656c038d38a09b28940768a2abc2d02e918aad
Jul 17 09:54:25.640741 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:25.640660    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp" event={"ID":"81815632-0afd-4111-bf76-808daa743644","Type":"ContainerStarted","Data":"03e93882a4dfc74c46252279aa656c038d38a09b28940768a2abc2d02e918aad"}
Jul 17 09:54:26.647864 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:26.647826    2571 generic.go:356] "Generic (PLEG): container finished" podID="81815632-0afd-4111-bf76-808daa743644" containerID="bd0f0cd1b9433383fe21ff2eb1d843658ddb9fe658e8157b09eeb88128688c98" exitCode=6
Jul 17 09:54:26.648271 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:26.647908    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp" event={"ID":"81815632-0afd-4111-bf76-808daa743644","Type":"ContainerDied","Data":"bd0f0cd1b9433383fe21ff2eb1d843658ddb9fe658e8157b09eeb88128688c98"}
Jul 17 09:54:26.648271 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:26.648090    2571 scope.go:117] "RemoveContainer" containerID="bd0f0cd1b9433383fe21ff2eb1d843658ddb9fe658e8157b09eeb88128688c98"
Jul 17 09:54:27.655574 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:27.655485    2571 generic.go:356] "Generic (PLEG): container finished" podID="81815632-0afd-4111-bf76-808daa743644" containerID="99cc200a9e18647b054fd4cc47866122224d40cc97aaf49151790a93c5e35553" exitCode=6
Jul 17 09:54:27.655574 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:27.655562    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp" event={"ID":"81815632-0afd-4111-bf76-808daa743644","Type":"ContainerDied","Data":"99cc200a9e18647b054fd4cc47866122224d40cc97aaf49151790a93c5e35553"}
Jul 17 09:54:27.656027 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:27.655623    2571 scope.go:117] "RemoveContainer" containerID="bd0f0cd1b9433383fe21ff2eb1d843658ddb9fe658e8157b09eeb88128688c98"
Jul 17 09:54:27.656027 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:27.655853    2571 scope.go:117] "RemoveContainer" containerID="99cc200a9e18647b054fd4cc47866122224d40cc97aaf49151790a93c5e35553"
Jul 17 09:54:27.656127 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:54:27.656052    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp_odh-ai-gateway-infra(81815632-0afd-4111-bf76-808daa743644)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp" podUID="81815632-0afd-4111-bf76-808daa743644"
Jul 17 09:54:28.663433 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:28.663400    2571 scope.go:117] "RemoveContainer" containerID="99cc200a9e18647b054fd4cc47866122224d40cc97aaf49151790a93c5e35553"
Jul 17 09:54:28.663825 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:54:28.663708    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp_odh-ai-gateway-infra(81815632-0afd-4111-bf76-808daa743644)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp" podUID="81815632-0afd-4111-bf76-808daa743644"
Jul 17 09:54:30.752811 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:30.752781    2571 scope.go:117] "RemoveContainer" containerID="c41adc71b076b48cf908f2f1d1f00165536017b474d8603d895b4508955153d2"
Jul 17 09:54:31.682475 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:31.682371    2571 generic.go:356] "Generic (PLEG): container finished" podID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerID="903933cc646521de549f3173bf50700aa0041d1f3adbca65abaaa75acd0842c9" exitCode=7
Jul 17 09:54:31.682651 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:31.682450    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5" event={"ID":"a6a2d033-6177-41d7-adc0-19e292c20d9e","Type":"ContainerDied","Data":"903933cc646521de549f3173bf50700aa0041d1f3adbca65abaaa75acd0842c9"}
Jul 17 09:54:31.682651 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:31.682599    2571 scope.go:117] "RemoveContainer" containerID="c41adc71b076b48cf908f2f1d1f00165536017b474d8603d895b4508955153d2"
Jul 17 09:54:31.682952 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:31.682927    2571 scope.go:117] "RemoveContainer" containerID="903933cc646521de549f3173bf50700aa0041d1f3adbca65abaaa75acd0842c9"
Jul 17 09:54:31.683186 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:54:31.683166    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5_odh-ai-gateway-infra(a6a2d033-6177-41d7-adc0-19e292c20d9e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e"
Jul 17 09:54:32.713556 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:32.713519    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"]
Jul 17 09:54:32.860108 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:32.860084    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"
Jul 17 09:54:32.975271 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:32.975183    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a2d033-6177-41d7-adc0-19e292c20d9e-maas-api-service-ca\") pod \"a6a2d033-6177-41d7-adc0-19e292c20d9e\" (UID: \"a6a2d033-6177-41d7-adc0-19e292c20d9e\") "
Jul 17 09:54:32.975636 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:32.975615    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6a2d033-6177-41d7-adc0-19e292c20d9e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a6a2d033-6177-41d7-adc0-19e292c20d9e" (UID: "a6a2d033-6177-41d7-adc0-19e292c20d9e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:54:33.076632 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:33.076584    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a2d033-6177-41d7-adc0-19e292c20d9e-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:54:33.696667 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:33.696628    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5" event={"ID":"a6a2d033-6177-41d7-adc0-19e292c20d9e","Type":"ContainerDied","Data":"692c669843112c0ac3905ab25ae47d6b92e236a6aa0d3178fed68718f3685048"}
Jul 17 09:54:33.696667 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:33.696661    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"
Jul 17 09:54:33.696930 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:33.696690    2571 scope.go:117] "RemoveContainer" containerID="903933cc646521de549f3173bf50700aa0041d1f3adbca65abaaa75acd0842c9"
Jul 17 09:54:33.719981 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:33.719949    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"]
Jul 17 09:54:33.725420 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:33.725385    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pdxk5"]
Jul 17 09:54:33.758260 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:33.758229    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e" path="/var/lib/kubelet/pods/a6a2d033-6177-41d7-adc0-19e292c20d9e/volumes"
Jul 17 09:54:41.752162 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:41.752126    2571 scope.go:117] "RemoveContainer" containerID="99cc200a9e18647b054fd4cc47866122224d40cc97aaf49151790a93c5e35553"
Jul 17 09:54:42.753865 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:42.753832    2571 generic.go:356] "Generic (PLEG): container finished" podID="81815632-0afd-4111-bf76-808daa743644" containerID="b67b01ebde7591dfaab13a5903ac3a124ad5a66a75d5cf9f27b04eddd269b5ee" exitCode=6
Jul 17 09:54:42.754330 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:42.753911    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp" event={"ID":"81815632-0afd-4111-bf76-808daa743644","Type":"ContainerDied","Data":"b67b01ebde7591dfaab13a5903ac3a124ad5a66a75d5cf9f27b04eddd269b5ee"}
Jul 17 09:54:42.754330 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:42.753979    2571 scope.go:117] "RemoveContainer" containerID="99cc200a9e18647b054fd4cc47866122224d40cc97aaf49151790a93c5e35553"
Jul 17 09:54:42.754330 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:42.754292    2571 scope.go:117] "RemoveContainer" containerID="b67b01ebde7591dfaab13a5903ac3a124ad5a66a75d5cf9f27b04eddd269b5ee"
Jul 17 09:54:42.754501 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:54:42.754483    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp_odh-ai-gateway-infra(81815632-0afd-4111-bf76-808daa743644)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp" podUID="81815632-0afd-4111-bf76-808daa743644"
Jul 17 09:54:43.783689 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:43.783653    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"]
Jul 17 09:54:43.927624 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:43.927587    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"
Jul 17 09:54:44.077246 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:44.077147    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81815632-0afd-4111-bf76-808daa743644-maas-api-service-ca\") pod \"81815632-0afd-4111-bf76-808daa743644\" (UID: \"81815632-0afd-4111-bf76-808daa743644\") "
Jul 17 09:54:44.077572 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:44.077549    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81815632-0afd-4111-bf76-808daa743644-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "81815632-0afd-4111-bf76-808daa743644" (UID: "81815632-0afd-4111-bf76-808daa743644"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:54:44.178476 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:44.178426    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81815632-0afd-4111-bf76-808daa743644-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:54:44.767924 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:44.767894    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"
Jul 17 09:54:44.768084 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:44.767938    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp" event={"ID":"81815632-0afd-4111-bf76-808daa743644","Type":"ContainerDied","Data":"03e93882a4dfc74c46252279aa656c038d38a09b28940768a2abc2d02e918aad"}
Jul 17 09:54:44.768084 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:44.768000    2571 scope.go:117] "RemoveContainer" containerID="b67b01ebde7591dfaab13a5903ac3a124ad5a66a75d5cf9f27b04eddd269b5ee"
Jul 17 09:54:44.796810 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:44.796771    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"]
Jul 17 09:54:44.800936 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:44.800908    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hhffp"]
Jul 17 09:54:45.758645 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:45.758610    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81815632-0afd-4111-bf76-808daa743644" path="/var/lib/kubelet/pods/81815632-0afd-4111-bf76-808daa743644/volumes"
Jul 17 09:54:54.647578 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.647536    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"]
Jul 17 09:54:54.648002 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.647987    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerName="revoke-keys"
Jul 17 09:54:54.648047 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648004    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f43c45-c706-4f66-a0c3-5b4b11eca5a5" containerName="revoke-keys"
Jul 17 09:54:54.648047 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648015    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81815632-0afd-4111-bf76-808daa743644" containerName="revoke-keys"
Jul 17 09:54:54.648047 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648020    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="81815632-0afd-4111-bf76-808daa743644" containerName="revoke-keys"
Jul 17 09:54:54.648047 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648027    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerName="revoke-keys"
Jul 17 09:54:54.648047 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648033    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerName="revoke-keys"
Jul 17 09:54:54.648206 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648055    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81815632-0afd-4111-bf76-808daa743644" containerName="revoke-keys"
Jul 17 09:54:54.648206 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648060    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="81815632-0afd-4111-bf76-808daa743644" containerName="revoke-keys"
Jul 17 09:54:54.648206 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648067    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerName="revoke-keys"
Jul 17 09:54:54.648206 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648072    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerName="revoke-keys"
Jul 17 09:54:54.648206 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648132    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="81815632-0afd-4111-bf76-808daa743644" containerName="revoke-keys"
Jul 17 09:54:54.648206 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648140    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerName="revoke-keys"
Jul 17 09:54:54.648206 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648147    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerName="revoke-keys"
Jul 17 09:54:54.648206 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648153    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="81815632-0afd-4111-bf76-808daa743644" containerName="revoke-keys"
Jul 17 09:54:54.648206 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.648158    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerName="revoke-keys"
Jul 17 09:54:54.652988 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.652968    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"
Jul 17 09:54:54.656393 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.656371    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:54:54.663746 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.663716    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"]
Jul 17 09:54:54.777179 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.777136    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38d66f80-d079-4850-8b8b-2108d0c53cca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6\" (UID: \"38d66f80-d079-4850-8b8b-2108d0c53cca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"
Jul 17 09:54:54.878060 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.878015    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38d66f80-d079-4850-8b8b-2108d0c53cca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6\" (UID: \"38d66f80-d079-4850-8b8b-2108d0c53cca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"
Jul 17 09:54:54.878766 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.878746    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38d66f80-d079-4850-8b8b-2108d0c53cca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6\" (UID: \"38d66f80-d079-4850-8b8b-2108d0c53cca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"
Jul 17 09:54:54.964731 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:54.964694    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"
Jul 17 09:54:55.114782 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:55.114744    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"]
Jul 17 09:54:55.114935 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:54:55.114777    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d66f80_d079_4850_8b8b_2108d0c53cca.slice/crio-f46b2485cd25e77879b84203de35c520ff931809cf3523bf4777155e1a1b089f WatchSource:0}: Error finding container f46b2485cd25e77879b84203de35c520ff931809cf3523bf4777155e1a1b089f: Status 404 returned error can't find the container with id f46b2485cd25e77879b84203de35c520ff931809cf3523bf4777155e1a1b089f
Jul 17 09:54:55.833908 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:55.833874    2571 generic.go:356] "Generic (PLEG): container finished" podID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerID="68994a3524f5c4d9ee18c3be5ac79a9d644772424dba9a0a9ea2e9784605ef57" exitCode=6
Jul 17 09:54:55.834341 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:55.833968    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6" event={"ID":"38d66f80-d079-4850-8b8b-2108d0c53cca","Type":"ContainerDied","Data":"68994a3524f5c4d9ee18c3be5ac79a9d644772424dba9a0a9ea2e9784605ef57"}
Jul 17 09:54:55.834341 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:55.834019    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6" event={"ID":"38d66f80-d079-4850-8b8b-2108d0c53cca","Type":"ContainerStarted","Data":"f46b2485cd25e77879b84203de35c520ff931809cf3523bf4777155e1a1b089f"}
Jul 17 09:54:55.834341 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:55.834150    2571 scope.go:117] "RemoveContainer" containerID="68994a3524f5c4d9ee18c3be5ac79a9d644772424dba9a0a9ea2e9784605ef57"
Jul 17 09:54:56.841121 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:56.841080    2571 generic.go:356] "Generic (PLEG): container finished" podID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerID="bc67c4f5cde8d12eafaa975b0ebaafc0984598e04c09007fafada95d53f2f182" exitCode=6
Jul 17 09:54:56.841586 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:56.841151    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6" event={"ID":"38d66f80-d079-4850-8b8b-2108d0c53cca","Type":"ContainerDied","Data":"bc67c4f5cde8d12eafaa975b0ebaafc0984598e04c09007fafada95d53f2f182"}
Jul 17 09:54:56.841586 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:56.841213    2571 scope.go:117] "RemoveContainer" containerID="68994a3524f5c4d9ee18c3be5ac79a9d644772424dba9a0a9ea2e9784605ef57"
Jul 17 09:54:56.841586 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:56.841363    2571 scope.go:117] "RemoveContainer" containerID="bc67c4f5cde8d12eafaa975b0ebaafc0984598e04c09007fafada95d53f2f182"
Jul 17 09:54:56.841719 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:54:56.841632    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6_odh-ai-gateway-infra(38d66f80-d079-4850-8b8b-2108d0c53cca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca"
Jul 17 09:54:57.854290 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:54:57.854260    2571 scope.go:117] "RemoveContainer" containerID="bc67c4f5cde8d12eafaa975b0ebaafc0984598e04c09007fafada95d53f2f182"
Jul 17 09:54:57.854709 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:54:57.854482    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6_odh-ai-gateway-infra(38d66f80-d079-4850-8b8b-2108d0c53cca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca"
Jul 17 09:55:11.752172 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:11.752041    2571 scope.go:117] "RemoveContainer" containerID="bc67c4f5cde8d12eafaa975b0ebaafc0984598e04c09007fafada95d53f2f182"
Jul 17 09:55:12.937504 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:12.937456    2571 generic.go:356] "Generic (PLEG): container finished" podID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerID="5f7a463fc5749e32a7d82d385d54a98786e8acd4d814909351df39d09bd28444" exitCode=6
Jul 17 09:55:12.937980 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:12.937536    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6" event={"ID":"38d66f80-d079-4850-8b8b-2108d0c53cca","Type":"ContainerDied","Data":"5f7a463fc5749e32a7d82d385d54a98786e8acd4d814909351df39d09bd28444"}
Jul 17 09:55:12.937980 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:12.937613    2571 scope.go:117] "RemoveContainer" containerID="bc67c4f5cde8d12eafaa975b0ebaafc0984598e04c09007fafada95d53f2f182"
Jul 17 09:55:12.937980 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:12.937864    2571 scope.go:117] "RemoveContainer" containerID="5f7a463fc5749e32a7d82d385d54a98786e8acd4d814909351df39d09bd28444"
Jul 17 09:55:12.938119 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:55:12.938093    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6_odh-ai-gateway-infra(38d66f80-d079-4850-8b8b-2108d0c53cca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca"
Jul 17 09:55:13.662329 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.662298    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"]
Jul 17 09:55:13.662750 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.662737    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81815632-0afd-4111-bf76-808daa743644" containerName="revoke-keys"
Jul 17 09:55:13.662804 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.662752    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="81815632-0afd-4111-bf76-808daa743644" containerName="revoke-keys"
Jul 17 09:55:13.662804 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.662761    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerName="revoke-keys"
Jul 17 09:55:13.662804 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.662766    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6a2d033-6177-41d7-adc0-19e292c20d9e" containerName="revoke-keys"
Jul 17 09:55:13.662900 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.662850    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="81815632-0afd-4111-bf76-808daa743644" containerName="revoke-keys"
Jul 17 09:55:13.667019 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.667003    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"
Jul 17 09:55:13.674734 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.674711    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"]
Jul 17 09:55:13.738994 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.738943    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd648e1-4b08-4795-ba4f-cadc892bec09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk\" (UID: \"ddd648e1-4b08-4795-ba4f-cadc892bec09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"
Jul 17 09:55:13.840308 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.840277    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd648e1-4b08-4795-ba4f-cadc892bec09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk\" (UID: \"ddd648e1-4b08-4795-ba4f-cadc892bec09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"
Jul 17 09:55:13.840922 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.840904    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd648e1-4b08-4795-ba4f-cadc892bec09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk\" (UID: \"ddd648e1-4b08-4795-ba4f-cadc892bec09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"
Jul 17 09:55:13.967271 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.967227    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"]
Jul 17 09:55:13.979879 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:13.979852    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"
Jul 17 09:55:14.144764 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.144732    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"
Jul 17 09:55:14.174348 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.174240    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"]
Jul 17 09:55:14.178017 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:55:14.177984    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd648e1_4b08_4795_ba4f_cadc892bec09.slice/crio-ac780cfa55aa3cae240ad07287bf71c7f50f363219d0f03b41d07fdb53093546 WatchSource:0}: Error finding container ac780cfa55aa3cae240ad07287bf71c7f50f363219d0f03b41d07fdb53093546: Status 404 returned error can't find the container with id ac780cfa55aa3cae240ad07287bf71c7f50f363219d0f03b41d07fdb53093546
Jul 17 09:55:14.244231 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.244201    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38d66f80-d079-4850-8b8b-2108d0c53cca-maas-api-service-ca\") pod \"38d66f80-d079-4850-8b8b-2108d0c53cca\" (UID: \"38d66f80-d079-4850-8b8b-2108d0c53cca\") "
Jul 17 09:55:14.244605 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.244582    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38d66f80-d079-4850-8b8b-2108d0c53cca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38d66f80-d079-4850-8b8b-2108d0c53cca" (UID: "38d66f80-d079-4850-8b8b-2108d0c53cca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:55:14.345807 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.345778    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38d66f80-d079-4850-8b8b-2108d0c53cca-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:55:14.951589 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.951558    2571 generic.go:356] "Generic (PLEG): container finished" podID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerID="d6ac8984f0303843e50800db645f8994f601074c6121b26ca0fbb4c987cc8605" exitCode=7
Jul 17 09:55:14.951744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.951641    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk" event={"ID":"ddd648e1-4b08-4795-ba4f-cadc892bec09","Type":"ContainerDied","Data":"d6ac8984f0303843e50800db645f8994f601074c6121b26ca0fbb4c987cc8605"}
Jul 17 09:55:14.951744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.951700    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk" event={"ID":"ddd648e1-4b08-4795-ba4f-cadc892bec09","Type":"ContainerStarted","Data":"ac780cfa55aa3cae240ad07287bf71c7f50f363219d0f03b41d07fdb53093546"}
Jul 17 09:55:14.951889 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.951830    2571 scope.go:117] "RemoveContainer" containerID="d6ac8984f0303843e50800db645f8994f601074c6121b26ca0fbb4c987cc8605"
Jul 17 09:55:14.953136 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.953114    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6" event={"ID":"38d66f80-d079-4850-8b8b-2108d0c53cca","Type":"ContainerDied","Data":"f46b2485cd25e77879b84203de35c520ff931809cf3523bf4777155e1a1b089f"}
Jul 17 09:55:14.953236 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.953138    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"
Jul 17 09:55:14.953236 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.953156    2571 scope.go:117] "RemoveContainer" containerID="5f7a463fc5749e32a7d82d385d54a98786e8acd4d814909351df39d09bd28444"
Jul 17 09:55:14.996972 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.996943    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"]
Jul 17 09:55:14.997383 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.997369    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerName="revoke-keys"
Jul 17 09:55:14.997383 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.997382    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerName="revoke-keys"
Jul 17 09:55:14.997506 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.997407    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerName="revoke-keys"
Jul 17 09:55:14.997506 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.997413    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerName="revoke-keys"
Jul 17 09:55:14.997506 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.997484    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerName="revoke-keys"
Jul 17 09:55:14.997506 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.997493    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerName="revoke-keys"
Jul 17 09:55:14.997506 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:14.997501    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerName="revoke-keys"
Jul 17 09:55:15.001723 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.001699    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"
Jul 17 09:55:15.009972 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.009944    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"]
Jul 17 09:55:15.042661 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.042629    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"]
Jul 17 09:55:15.050593 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.050556    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-6lst6"]
Jul 17 09:55:15.053248 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.053215    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e9da3b-9ada-47a0-81ab-b3935d3db645-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k\" (UID: \"78e9da3b-9ada-47a0-81ab-b3935d3db645\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"
Jul 17 09:55:15.154129 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.154090    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e9da3b-9ada-47a0-81ab-b3935d3db645-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k\" (UID: \"78e9da3b-9ada-47a0-81ab-b3935d3db645\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"
Jul 17 09:55:15.154805 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.154782    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e9da3b-9ada-47a0-81ab-b3935d3db645-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k\" (UID: \"78e9da3b-9ada-47a0-81ab-b3935d3db645\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"
Jul 17 09:55:15.328025 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.327988    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"
Jul 17 09:55:15.471620 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.471589    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"]
Jul 17 09:55:15.473320 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:55:15.473286    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78e9da3b_9ada_47a0_81ab_b3935d3db645.slice/crio-af237ebb41db626120c98169ec1fae2dfa79068e20e232240f29ec40753f7654 WatchSource:0}: Error finding container af237ebb41db626120c98169ec1fae2dfa79068e20e232240f29ec40753f7654: Status 404 returned error can't find the container with id af237ebb41db626120c98169ec1fae2dfa79068e20e232240f29ec40753f7654
Jul 17 09:55:15.759139 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.759107    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca" path="/var/lib/kubelet/pods/38d66f80-d079-4850-8b8b-2108d0c53cca/volumes"
Jul 17 09:55:15.961475 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.961434    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k" event={"ID":"78e9da3b-9ada-47a0-81ab-b3935d3db645","Type":"ContainerStarted","Data":"11d1b490a6cda69ffc7e6ae559af2e3cd9dcefac3148b95727a2d9fac684774a"}
Jul 17 09:55:15.961665 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.961511    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k" event={"ID":"78e9da3b-9ada-47a0-81ab-b3935d3db645","Type":"ContainerStarted","Data":"af237ebb41db626120c98169ec1fae2dfa79068e20e232240f29ec40753f7654"}
Jul 17 09:55:15.963190 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.963166    2571 generic.go:356] "Generic (PLEG): container finished" podID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerID="036f3071a5ea73af3cf45c8c2603ef1e41cca97516c956373f5b7e0fae918b60" exitCode=7
Jul 17 09:55:15.963297 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.963207    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk" event={"ID":"ddd648e1-4b08-4795-ba4f-cadc892bec09","Type":"ContainerDied","Data":"036f3071a5ea73af3cf45c8c2603ef1e41cca97516c956373f5b7e0fae918b60"}
Jul 17 09:55:15.963297 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.963253    2571 scope.go:117] "RemoveContainer" containerID="d6ac8984f0303843e50800db645f8994f601074c6121b26ca0fbb4c987cc8605"
Jul 17 09:55:15.963414 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.963395    2571 scope.go:117] "RemoveContainer" containerID="036f3071a5ea73af3cf45c8c2603ef1e41cca97516c956373f5b7e0fae918b60"
Jul 17 09:55:15.963623 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:55:15.963604    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk_odh-ai-gateway-infra(ddd648e1-4b08-4795-ba4f-cadc892bec09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09"
Jul 17 09:55:15.980011 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:15.979973    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k" podStartSLOduration=1.979958854 podStartE2EDuration="1.979958854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:55:15.979144206 +0000 UTC m=+2200.043964930" watchObservedRunningTime="2026-07-17 09:55:15.979958854 +0000 UTC m=+2200.044779576"
Jul 17 09:55:16.973606 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:16.973576    2571 generic.go:356] "Generic (PLEG): container finished" podID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerID="11d1b490a6cda69ffc7e6ae559af2e3cd9dcefac3148b95727a2d9fac684774a" exitCode=6
Jul 17 09:55:16.974098 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:16.973660    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k" event={"ID":"78e9da3b-9ada-47a0-81ab-b3935d3db645","Type":"ContainerDied","Data":"11d1b490a6cda69ffc7e6ae559af2e3cd9dcefac3148b95727a2d9fac684774a"}
Jul 17 09:55:16.974098 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:16.973834    2571 scope.go:117] "RemoveContainer" containerID="11d1b490a6cda69ffc7e6ae559af2e3cd9dcefac3148b95727a2d9fac684774a"
Jul 17 09:55:16.975549 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:16.975533    2571 scope.go:117] "RemoveContainer" containerID="036f3071a5ea73af3cf45c8c2603ef1e41cca97516c956373f5b7e0fae918b60"
Jul 17 09:55:16.975730 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:55:16.975710    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk_odh-ai-gateway-infra(ddd648e1-4b08-4795-ba4f-cadc892bec09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09"
Jul 17 09:55:17.982304 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:17.982274    2571 generic.go:356] "Generic (PLEG): container finished" podID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerID="7883edb731a5dcd81c77f1a9ad95a4f95f83f08b09d88aaea8cbc8846ed904ed" exitCode=6
Jul 17 09:55:17.982712 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:17.982369    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k" event={"ID":"78e9da3b-9ada-47a0-81ab-b3935d3db645","Type":"ContainerDied","Data":"7883edb731a5dcd81c77f1a9ad95a4f95f83f08b09d88aaea8cbc8846ed904ed"}
Jul 17 09:55:17.982712 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:17.982425    2571 scope.go:117] "RemoveContainer" containerID="11d1b490a6cda69ffc7e6ae559af2e3cd9dcefac3148b95727a2d9fac684774a"
Jul 17 09:55:17.982712 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:17.982543    2571 scope.go:117] "RemoveContainer" containerID="7883edb731a5dcd81c77f1a9ad95a4f95f83f08b09d88aaea8cbc8846ed904ed"
Jul 17 09:55:17.982712 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:55:17.982704    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k_odh-ai-gateway-infra(78e9da3b-9ada-47a0-81ab-b3935d3db645)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645"
Jul 17 09:55:18.990224 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:18.990195    2571 scope.go:117] "RemoveContainer" containerID="7883edb731a5dcd81c77f1a9ad95a4f95f83f08b09d88aaea8cbc8846ed904ed"
Jul 17 09:55:18.990590 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:55:18.990371    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k_odh-ai-gateway-infra(78e9da3b-9ada-47a0-81ab-b3935d3db645)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645"
Jul 17 09:55:28.752930 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:28.752901    2571 scope.go:117] "RemoveContainer" containerID="036f3071a5ea73af3cf45c8c2603ef1e41cca97516c956373f5b7e0fae918b60"
Jul 17 09:55:29.050852 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:29.050825    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk" event={"ID":"ddd648e1-4b08-4795-ba4f-cadc892bec09","Type":"ContainerStarted","Data":"114a203bbf51b5e929bfc2ffe220a9e8a236d0c0d694ab8081f122785bc4c1c6"}
Jul 17 09:55:29.072833 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:29.072783    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk" podStartSLOduration=16.072767519 podStartE2EDuration="16.072767519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:55:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:55:29.070341253 +0000 UTC m=+2213.135161978" watchObservedRunningTime="2026-07-17 09:55:29.072767519 +0000 UTC m=+2213.137588241"
Jul 17 09:55:30.058268 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:30.058233    2571 generic.go:356] "Generic (PLEG): container finished" podID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerID="114a203bbf51b5e929bfc2ffe220a9e8a236d0c0d694ab8081f122785bc4c1c6" exitCode=7
Jul 17 09:55:30.058681 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:30.058306    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk" event={"ID":"ddd648e1-4b08-4795-ba4f-cadc892bec09","Type":"ContainerDied","Data":"114a203bbf51b5e929bfc2ffe220a9e8a236d0c0d694ab8081f122785bc4c1c6"}
Jul 17 09:55:30.058681 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:30.058366    2571 scope.go:117] "RemoveContainer" containerID="036f3071a5ea73af3cf45c8c2603ef1e41cca97516c956373f5b7e0fae918b60"
Jul 17 09:55:30.058825 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:30.058727    2571 scope.go:117] "RemoveContainer" containerID="114a203bbf51b5e929bfc2ffe220a9e8a236d0c0d694ab8081f122785bc4c1c6"
Jul 17 09:55:30.059006 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:55:30.058983    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk_odh-ai-gateway-infra(ddd648e1-4b08-4795-ba4f-cadc892bec09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09"
Jul 17 09:55:30.087196 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:30.087162    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"]
Jul 17 09:55:31.205698 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:31.205673    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"
Jul 17 09:55:31.306841 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:31.306794    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd648e1-4b08-4795-ba4f-cadc892bec09-maas-api-service-ca\") pod \"ddd648e1-4b08-4795-ba4f-cadc892bec09\" (UID: \"ddd648e1-4b08-4795-ba4f-cadc892bec09\") "
Jul 17 09:55:31.307176 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:31.307151    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddd648e1-4b08-4795-ba4f-cadc892bec09-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ddd648e1-4b08-4795-ba4f-cadc892bec09" (UID: "ddd648e1-4b08-4795-ba4f-cadc892bec09"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:55:31.408662 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:31.408576    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddd648e1-4b08-4795-ba4f-cadc892bec09-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:55:32.073685 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:32.073605    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"
Jul 17 09:55:32.073685 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:32.073640    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk" event={"ID":"ddd648e1-4b08-4795-ba4f-cadc892bec09","Type":"ContainerDied","Data":"ac780cfa55aa3cae240ad07287bf71c7f50f363219d0f03b41d07fdb53093546"}
Jul 17 09:55:32.073886 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:32.073703    2571 scope.go:117] "RemoveContainer" containerID="114a203bbf51b5e929bfc2ffe220a9e8a236d0c0d694ab8081f122785bc4c1c6"
Jul 17 09:55:32.096824 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:32.096796    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"]
Jul 17 09:55:32.104527 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:32.104495    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d2srk"]
Jul 17 09:55:32.752256 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:32.752229    2571 scope.go:117] "RemoveContainer" containerID="7883edb731a5dcd81c77f1a9ad95a4f95f83f08b09d88aaea8cbc8846ed904ed"
Jul 17 09:55:33.081967 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:33.081928    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k" event={"ID":"78e9da3b-9ada-47a0-81ab-b3935d3db645","Type":"ContainerStarted","Data":"9a33e587548dcae8b98d51d30bb14911f48d97a1790ca8275bcd9965ead6cb4f"}
Jul 17 09:55:33.759352 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:33.759318    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09" path="/var/lib/kubelet/pods/ddd648e1-4b08-4795-ba4f-cadc892bec09/volumes"
Jul 17 09:55:34.090771 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:34.090688    2571 generic.go:356] "Generic (PLEG): container finished" podID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerID="9a33e587548dcae8b98d51d30bb14911f48d97a1790ca8275bcd9965ead6cb4f" exitCode=6
Jul 17 09:55:34.090902 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:34.090766    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k" event={"ID":"78e9da3b-9ada-47a0-81ab-b3935d3db645","Type":"ContainerDied","Data":"9a33e587548dcae8b98d51d30bb14911f48d97a1790ca8275bcd9965ead6cb4f"}
Jul 17 09:55:34.090902 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:34.090831    2571 scope.go:117] "RemoveContainer" containerID="7883edb731a5dcd81c77f1a9ad95a4f95f83f08b09d88aaea8cbc8846ed904ed"
Jul 17 09:55:34.091086 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:34.091070    2571 scope.go:117] "RemoveContainer" containerID="9a33e587548dcae8b98d51d30bb14911f48d97a1790ca8275bcd9965ead6cb4f"
Jul 17 09:55:34.091307 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:55:34.091289    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k_odh-ai-gateway-infra(78e9da3b-9ada-47a0-81ab-b3935d3db645)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645"
Jul 17 09:55:34.119043 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:34.119012    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"]
Jul 17 09:55:35.244794 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:35.244761    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"
Jul 17 09:55:35.346216 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:35.346184    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e9da3b-9ada-47a0-81ab-b3935d3db645-maas-api-service-ca\") pod \"78e9da3b-9ada-47a0-81ab-b3935d3db645\" (UID: \"78e9da3b-9ada-47a0-81ab-b3935d3db645\") "
Jul 17 09:55:35.346613 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:35.346592    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78e9da3b-9ada-47a0-81ab-b3935d3db645-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "78e9da3b-9ada-47a0-81ab-b3935d3db645" (UID: "78e9da3b-9ada-47a0-81ab-b3935d3db645"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:55:35.447684 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:35.447652    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78e9da3b-9ada-47a0-81ab-b3935d3db645-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:55:36.106204 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:36.106121    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"
Jul 17 09:55:36.106363 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:36.106127    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k" event={"ID":"78e9da3b-9ada-47a0-81ab-b3935d3db645","Type":"ContainerDied","Data":"af237ebb41db626120c98169ec1fae2dfa79068e20e232240f29ec40753f7654"}
Jul 17 09:55:36.106363 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:36.106258    2571 scope.go:117] "RemoveContainer" containerID="9a33e587548dcae8b98d51d30bb14911f48d97a1790ca8275bcd9965ead6cb4f"
Jul 17 09:55:36.129003 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:36.128975    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"]
Jul 17 09:55:36.137539 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:36.137511    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gtp4k"]
Jul 17 09:55:37.758962 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:37.758926    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645" path="/var/lib/kubelet/pods/78e9da3b-9ada-47a0-81ab-b3935d3db645/volumes"
Jul 17 09:55:54.698182 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698151    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"]
Jul 17 09:55:54.698668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698596    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerName="revoke-keys"
Jul 17 09:55:54.698668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698607    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerName="revoke-keys"
Jul 17 09:55:54.698668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698616    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerName="revoke-keys"
Jul 17 09:55:54.698668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698623    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerName="revoke-keys"
Jul 17 09:55:54.698668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698632    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerName="revoke-keys"
Jul 17 09:55:54.698668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698638    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerName="revoke-keys"
Jul 17 09:55:54.698668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698648    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerName="revoke-keys"
Jul 17 09:55:54.698668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698653    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="38d66f80-d079-4850-8b8b-2108d0c53cca" containerName="revoke-keys"
Jul 17 09:55:54.698668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698662    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerName="revoke-keys"
Jul 17 09:55:54.698668 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698669    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerName="revoke-keys"
Jul 17 09:55:54.699011 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698675    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerName="revoke-keys"
Jul 17 09:55:54.699011 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698681    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerName="revoke-keys"
Jul 17 09:55:54.699011 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698742    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerName="revoke-keys"
Jul 17 09:55:54.699011 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698748    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerName="revoke-keys"
Jul 17 09:55:54.699011 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698753    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerName="revoke-keys"
Jul 17 09:55:54.699011 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.698760    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerName="revoke-keys"
Jul 17 09:55:54.701981 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.701965    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"
Jul 17 09:55:54.705085 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.705065    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:55:54.710287 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.710030    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"]
Jul 17 09:55:54.720595 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.720568    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1662001-2d08-40d7-9a06-2e23e9f60145-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2\" (UID: \"b1662001-2d08-40d7-9a06-2e23e9f60145\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"
Jul 17 09:55:54.821568 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.821512    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1662001-2d08-40d7-9a06-2e23e9f60145-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2\" (UID: \"b1662001-2d08-40d7-9a06-2e23e9f60145\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"
Jul 17 09:55:54.822191 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:54.822169    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1662001-2d08-40d7-9a06-2e23e9f60145-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2\" (UID: \"b1662001-2d08-40d7-9a06-2e23e9f60145\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"
Jul 17 09:55:55.014579 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:55.014486    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"
Jul 17 09:55:55.152213 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:55.152184    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"]
Jul 17 09:55:55.154103 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:55:55.154073    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1662001_2d08_40d7_9a06_2e23e9f60145.slice/crio-60aaf07df675ab35ddb8a1fede65eb683f6a15562461e3f1fd5285a089c5d09f WatchSource:0}: Error finding container 60aaf07df675ab35ddb8a1fede65eb683f6a15562461e3f1fd5285a089c5d09f: Status 404 returned error can't find the container with id 60aaf07df675ab35ddb8a1fede65eb683f6a15562461e3f1fd5285a089c5d09f
Jul 17 09:55:55.220422 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:55.220395    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2" event={"ID":"b1662001-2d08-40d7-9a06-2e23e9f60145","Type":"ContainerStarted","Data":"60aaf07df675ab35ddb8a1fede65eb683f6a15562461e3f1fd5285a089c5d09f"}
Jul 17 09:55:56.227850 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:56.227813    2571 generic.go:356] "Generic (PLEG): container finished" podID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerID="d4bd2d207b44a240df44b4bc5f1da5b5a3d344286f49f59e5ff882d8013ef8e8" exitCode=6
Jul 17 09:55:56.228229 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:56.227896    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2" event={"ID":"b1662001-2d08-40d7-9a06-2e23e9f60145","Type":"ContainerDied","Data":"d4bd2d207b44a240df44b4bc5f1da5b5a3d344286f49f59e5ff882d8013ef8e8"}
Jul 17 09:55:56.228229 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:56.228092    2571 scope.go:117] "RemoveContainer" containerID="d4bd2d207b44a240df44b4bc5f1da5b5a3d344286f49f59e5ff882d8013ef8e8"
Jul 17 09:55:57.234554 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:57.234517    2571 generic.go:356] "Generic (PLEG): container finished" podID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerID="3e9bbd7d6484cabf92ca4cd7acc11ad79d22e53682becd4cb6b4b2e65707e991" exitCode=6
Jul 17 09:55:57.234996 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:57.234603    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2" event={"ID":"b1662001-2d08-40d7-9a06-2e23e9f60145","Type":"ContainerDied","Data":"3e9bbd7d6484cabf92ca4cd7acc11ad79d22e53682becd4cb6b4b2e65707e991"}
Jul 17 09:55:57.234996 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:57.234681    2571 scope.go:117] "RemoveContainer" containerID="d4bd2d207b44a240df44b4bc5f1da5b5a3d344286f49f59e5ff882d8013ef8e8"
Jul 17 09:55:57.234996 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:57.234791    2571 scope.go:117] "RemoveContainer" containerID="3e9bbd7d6484cabf92ca4cd7acc11ad79d22e53682becd4cb6b4b2e65707e991"
Jul 17 09:55:57.235121 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:55:57.235003    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2_odh-ai-gateway-infra(b1662001-2d08-40d7-9a06-2e23e9f60145)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145"
Jul 17 09:55:58.242291 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:55:58.242210    2571 scope.go:117] "RemoveContainer" containerID="3e9bbd7d6484cabf92ca4cd7acc11ad79d22e53682becd4cb6b4b2e65707e991"
Jul 17 09:55:58.242693 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:55:58.242428    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2_odh-ai-gateway-infra(b1662001-2d08-40d7-9a06-2e23e9f60145)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145"
Jul 17 09:56:03.717650 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:03.717617    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"]
Jul 17 09:56:03.718082 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:03.718049    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerName="revoke-keys"
Jul 17 09:56:03.718082 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:03.718059    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerName="revoke-keys"
Jul 17 09:56:03.718153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:03.718127    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="78e9da3b-9ada-47a0-81ab-b3935d3db645" containerName="revoke-keys"
Jul 17 09:56:03.718153 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:03.718137    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd648e1-4b08-4795-ba4f-cadc892bec09" containerName="revoke-keys"
Jul 17 09:56:03.722695 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:03.722678    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"
Jul 17 09:56:03.734260 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:03.734232    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"]
Jul 17 09:56:03.802326 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:03.802270    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52d8d6a9-0d96-4cb1-8f53-2dd5474a1832-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd\" (UID: \"52d8d6a9-0d96-4cb1-8f53-2dd5474a1832\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"
Jul 17 09:56:03.903381 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:03.903340    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52d8d6a9-0d96-4cb1-8f53-2dd5474a1832-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd\" (UID: \"52d8d6a9-0d96-4cb1-8f53-2dd5474a1832\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"
Jul 17 09:56:03.903998 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:03.903976    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52d8d6a9-0d96-4cb1-8f53-2dd5474a1832-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd\" (UID: \"52d8d6a9-0d96-4cb1-8f53-2dd5474a1832\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"
Jul 17 09:56:04.033802 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:04.033720    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"
Jul 17 09:56:04.381895 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:04.381863    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"]
Jul 17 09:56:04.382528 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:56:04.382502    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d8d6a9_0d96_4cb1_8f53_2dd5474a1832.slice/crio-4cb2bc5c0357d75db5334fbb27b50ef4db0ec93db01a862eea52f1575650a740 WatchSource:0}: Error finding container 4cb2bc5c0357d75db5334fbb27b50ef4db0ec93db01a862eea52f1575650a740: Status 404 returned error can't find the container with id 4cb2bc5c0357d75db5334fbb27b50ef4db0ec93db01a862eea52f1575650a740
Jul 17 09:56:05.282745 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:05.282710    2571 generic.go:356] "Generic (PLEG): container finished" podID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerID="d736e385e3d9c1c626d8bb50cba3cb4710938f8501b8f1ebc6964d4961a23c87" exitCode=7
Jul 17 09:56:05.283155 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:05.282800    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd" event={"ID":"52d8d6a9-0d96-4cb1-8f53-2dd5474a1832","Type":"ContainerDied","Data":"d736e385e3d9c1c626d8bb50cba3cb4710938f8501b8f1ebc6964d4961a23c87"}
Jul 17 09:56:05.283155 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:05.282869    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd" event={"ID":"52d8d6a9-0d96-4cb1-8f53-2dd5474a1832","Type":"ContainerStarted","Data":"4cb2bc5c0357d75db5334fbb27b50ef4db0ec93db01a862eea52f1575650a740"}
Jul 17 09:56:05.283155 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:05.282958    2571 scope.go:117] "RemoveContainer" containerID="d736e385e3d9c1c626d8bb50cba3cb4710938f8501b8f1ebc6964d4961a23c87"
Jul 17 09:56:06.290738 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:06.290698    2571 generic.go:356] "Generic (PLEG): container finished" podID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerID="be58aec1bcd86d31efbf2de5b85340daed589c212612249d30f6dbfb5828e710" exitCode=7
Jul 17 09:56:06.291183 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:06.290785    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd" event={"ID":"52d8d6a9-0d96-4cb1-8f53-2dd5474a1832","Type":"ContainerDied","Data":"be58aec1bcd86d31efbf2de5b85340daed589c212612249d30f6dbfb5828e710"}
Jul 17 09:56:06.291183 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:06.290842    2571 scope.go:117] "RemoveContainer" containerID="d736e385e3d9c1c626d8bb50cba3cb4710938f8501b8f1ebc6964d4961a23c87"
Jul 17 09:56:06.291183 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:06.291081    2571 scope.go:117] "RemoveContainer" containerID="be58aec1bcd86d31efbf2de5b85340daed589c212612249d30f6dbfb5828e710"
Jul 17 09:56:06.291356 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:56:06.291285    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd_odh-ai-gateway-infra(52d8d6a9-0d96-4cb1-8f53-2dd5474a1832)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832"
Jul 17 09:56:07.299375 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:07.299347    2571 scope.go:117] "RemoveContainer" containerID="be58aec1bcd86d31efbf2de5b85340daed589c212612249d30f6dbfb5828e710"
Jul 17 09:56:07.299771 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:56:07.299597    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd_odh-ai-gateway-infra(52d8d6a9-0d96-4cb1-8f53-2dd5474a1832)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832"
Jul 17 09:56:09.753095 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:09.753062    2571 scope.go:117] "RemoveContainer" containerID="3e9bbd7d6484cabf92ca4cd7acc11ad79d22e53682becd4cb6b4b2e65707e991"
Jul 17 09:56:10.318257 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:10.318174    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2" event={"ID":"b1662001-2d08-40d7-9a06-2e23e9f60145","Type":"ContainerStarted","Data":"f7d5d487bafdb0f0810876c69ae24299d6213cd67d4837dd79407f1d3f8510cf"}
Jul 17 09:56:10.340539 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:10.340494    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2" podStartSLOduration=16.340477486 podStartE2EDuration="16.340477486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:55:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:56:10.336846992 +0000 UTC m=+2254.401667715" watchObservedRunningTime="2026-07-17 09:56:10.340477486 +0000 UTC m=+2254.405298205"
Jul 17 09:56:11.325936 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:11.325903    2571 generic.go:356] "Generic (PLEG): container finished" podID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerID="f7d5d487bafdb0f0810876c69ae24299d6213cd67d4837dd79407f1d3f8510cf" exitCode=6
Jul 17 09:56:11.326332 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:11.325957    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2" event={"ID":"b1662001-2d08-40d7-9a06-2e23e9f60145","Type":"ContainerDied","Data":"f7d5d487bafdb0f0810876c69ae24299d6213cd67d4837dd79407f1d3f8510cf"}
Jul 17 09:56:11.326332 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:11.326004    2571 scope.go:117] "RemoveContainer" containerID="3e9bbd7d6484cabf92ca4cd7acc11ad79d22e53682becd4cb6b4b2e65707e991"
Jul 17 09:56:11.326332 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:11.326318    2571 scope.go:117] "RemoveContainer" containerID="f7d5d487bafdb0f0810876c69ae24299d6213cd67d4837dd79407f1d3f8510cf"
Jul 17 09:56:11.326786 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:56:11.326558    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2_odh-ai-gateway-infra(b1662001-2d08-40d7-9a06-2e23e9f60145)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145"
Jul 17 09:56:11.351178 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:11.351150    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"]
Jul 17 09:56:12.475794 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:12.475761    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"
Jul 17 09:56:12.580563 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:12.580525    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1662001-2d08-40d7-9a06-2e23e9f60145-maas-api-service-ca\") pod \"b1662001-2d08-40d7-9a06-2e23e9f60145\" (UID: \"b1662001-2d08-40d7-9a06-2e23e9f60145\") "
Jul 17 09:56:12.580905 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:12.580878    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1662001-2d08-40d7-9a06-2e23e9f60145-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b1662001-2d08-40d7-9a06-2e23e9f60145" (UID: "b1662001-2d08-40d7-9a06-2e23e9f60145"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:56:12.681990 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:12.681898    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1662001-2d08-40d7-9a06-2e23e9f60145-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:56:13.340701 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:13.340669    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"
Jul 17 09:56:13.340917 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:13.340720    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2" event={"ID":"b1662001-2d08-40d7-9a06-2e23e9f60145","Type":"ContainerDied","Data":"60aaf07df675ab35ddb8a1fede65eb683f6a15562461e3f1fd5285a089c5d09f"}
Jul 17 09:56:13.340917 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:13.340792    2571 scope.go:117] "RemoveContainer" containerID="f7d5d487bafdb0f0810876c69ae24299d6213cd67d4837dd79407f1d3f8510cf"
Jul 17 09:56:13.367066 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:13.367035    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"]
Jul 17 09:56:13.372950 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:13.372923    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcmb2"]
Jul 17 09:56:13.758619 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:13.758589    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145" path="/var/lib/kubelet/pods/b1662001-2d08-40d7-9a06-2e23e9f60145/volumes"
Jul 17 09:56:15.063738 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.063702    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"]
Jul 17 09:56:15.064138 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.064125    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerName="revoke-keys"
Jul 17 09:56:15.064183 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.064140    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerName="revoke-keys"
Jul 17 09:56:15.064183 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.064150    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerName="revoke-keys"
Jul 17 09:56:15.064183 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.064155    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerName="revoke-keys"
Jul 17 09:56:15.064278 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.064230    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerName="revoke-keys"
Jul 17 09:56:15.064278 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.064243    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerName="revoke-keys"
Jul 17 09:56:15.068528 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.068513    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"
Jul 17 09:56:15.081030 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.081007    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"]
Jul 17 09:56:15.204544 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.204504    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f73f48ab-4405-44a0-a1be-09d07967f222-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg\" (UID: \"f73f48ab-4405-44a0-a1be-09d07967f222\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"
Jul 17 09:56:15.305497 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.305449    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f73f48ab-4405-44a0-a1be-09d07967f222-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg\" (UID: \"f73f48ab-4405-44a0-a1be-09d07967f222\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"
Jul 17 09:56:15.306123 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.306098    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f73f48ab-4405-44a0-a1be-09d07967f222-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg\" (UID: \"f73f48ab-4405-44a0-a1be-09d07967f222\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"
Jul 17 09:56:15.378172 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.378067    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"
Jul 17 09:56:15.529770 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:15.529741    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"]
Jul 17 09:56:15.531742 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:56:15.531705    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73f48ab_4405_44a0_a1be_09d07967f222.slice/crio-ce11c4fd0b8a0a9b7fc017b682c576271d1bcc7f7c4b325ed7a5c9708a35caa3 WatchSource:0}: Error finding container ce11c4fd0b8a0a9b7fc017b682c576271d1bcc7f7c4b325ed7a5c9708a35caa3: Status 404 returned error can't find the container with id ce11c4fd0b8a0a9b7fc017b682c576271d1bcc7f7c4b325ed7a5c9708a35caa3
Jul 17 09:56:16.360384 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:16.360351    2571 generic.go:356] "Generic (PLEG): container finished" podID="f73f48ab-4405-44a0-a1be-09d07967f222" containerID="222019a9447fa4bb1975e62e2746f5db81f5ca7df607b9bc9efea84a301517a7" exitCode=6
Jul 17 09:56:16.360856 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:16.360407    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg" event={"ID":"f73f48ab-4405-44a0-a1be-09d07967f222","Type":"ContainerDied","Data":"222019a9447fa4bb1975e62e2746f5db81f5ca7df607b9bc9efea84a301517a7"}
Jul 17 09:56:16.360856 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:16.360444    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg" event={"ID":"f73f48ab-4405-44a0-a1be-09d07967f222","Type":"ContainerStarted","Data":"ce11c4fd0b8a0a9b7fc017b682c576271d1bcc7f7c4b325ed7a5c9708a35caa3"}
Jul 17 09:56:16.360856 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:16.360603    2571 scope.go:117] "RemoveContainer" containerID="222019a9447fa4bb1975e62e2746f5db81f5ca7df607b9bc9efea84a301517a7"
Jul 17 09:56:17.368400 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:17.368302    2571 generic.go:356] "Generic (PLEG): container finished" podID="f73f48ab-4405-44a0-a1be-09d07967f222" containerID="45b13026f61efa8c0d97f038a03db21c53dedeb3bc7127053628f603a660fdbf" exitCode=6
Jul 17 09:56:17.368839 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:17.368388    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg" event={"ID":"f73f48ab-4405-44a0-a1be-09d07967f222","Type":"ContainerDied","Data":"45b13026f61efa8c0d97f038a03db21c53dedeb3bc7127053628f603a660fdbf"}
Jul 17 09:56:17.368839 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:17.368525    2571 scope.go:117] "RemoveContainer" containerID="222019a9447fa4bb1975e62e2746f5db81f5ca7df607b9bc9efea84a301517a7"
Jul 17 09:56:17.368839 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:17.368669    2571 scope.go:117] "RemoveContainer" containerID="45b13026f61efa8c0d97f038a03db21c53dedeb3bc7127053628f603a660fdbf"
Jul 17 09:56:17.368974 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:56:17.368910    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg_odh-ai-gateway-infra(f73f48ab-4405-44a0-a1be-09d07967f222)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg" podUID="f73f48ab-4405-44a0-a1be-09d07967f222"
Jul 17 09:56:18.375589 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:18.375560    2571 scope.go:117] "RemoveContainer" containerID="45b13026f61efa8c0d97f038a03db21c53dedeb3bc7127053628f603a660fdbf"
Jul 17 09:56:18.375978 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:56:18.375744    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg_odh-ai-gateway-infra(f73f48ab-4405-44a0-a1be-09d07967f222)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg" podUID="f73f48ab-4405-44a0-a1be-09d07967f222"
Jul 17 09:56:20.752018 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:20.751986    2571 scope.go:117] "RemoveContainer" containerID="be58aec1bcd86d31efbf2de5b85340daed589c212612249d30f6dbfb5828e710"
Jul 17 09:56:21.395536 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:21.395500    2571 generic.go:356] "Generic (PLEG): container finished" podID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerID="835e4f4a35fba84c4c2dad436631f7eb14acbe973b302bfb2871b6c77575a6c2" exitCode=7
Jul 17 09:56:21.395719 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:21.395569    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd" event={"ID":"52d8d6a9-0d96-4cb1-8f53-2dd5474a1832","Type":"ContainerDied","Data":"835e4f4a35fba84c4c2dad436631f7eb14acbe973b302bfb2871b6c77575a6c2"}
Jul 17 09:56:21.395719 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:21.395633    2571 scope.go:117] "RemoveContainer" containerID="be58aec1bcd86d31efbf2de5b85340daed589c212612249d30f6dbfb5828e710"
Jul 17 09:56:21.395974 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:21.395954    2571 scope.go:117] "RemoveContainer" containerID="835e4f4a35fba84c4c2dad436631f7eb14acbe973b302bfb2871b6c77575a6c2"
Jul 17 09:56:21.396206 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:56:21.396183    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd_odh-ai-gateway-infra(52d8d6a9-0d96-4cb1-8f53-2dd5474a1832)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832"
Jul 17 09:56:22.434197 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:22.434162    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"]
Jul 17 09:56:22.580045 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:22.580020    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"
Jul 17 09:56:22.674049 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:22.674010    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52d8d6a9-0d96-4cb1-8f53-2dd5474a1832-maas-api-service-ca\") pod \"52d8d6a9-0d96-4cb1-8f53-2dd5474a1832\" (UID: \"52d8d6a9-0d96-4cb1-8f53-2dd5474a1832\") "
Jul 17 09:56:22.674407 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:22.674381    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52d8d6a9-0d96-4cb1-8f53-2dd5474a1832-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" (UID: "52d8d6a9-0d96-4cb1-8f53-2dd5474a1832"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:56:22.775448 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:22.775401    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52d8d6a9-0d96-4cb1-8f53-2dd5474a1832-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:56:23.420597 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:23.420560    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd" event={"ID":"52d8d6a9-0d96-4cb1-8f53-2dd5474a1832","Type":"ContainerDied","Data":"4cb2bc5c0357d75db5334fbb27b50ef4db0ec93db01a862eea52f1575650a740"}
Jul 17 09:56:23.420597 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:23.420583    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"
Jul 17 09:56:23.420822 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:23.420626    2571 scope.go:117] "RemoveContainer" containerID="835e4f4a35fba84c4c2dad436631f7eb14acbe973b302bfb2871b6c77575a6c2"
Jul 17 09:56:23.447535 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:23.447505    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"]
Jul 17 09:56:23.449884 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:23.449863    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-78rrd"]
Jul 17 09:56:23.758705 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:23.758670    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" path="/var/lib/kubelet/pods/52d8d6a9-0d96-4cb1-8f53-2dd5474a1832/volumes"
Jul 17 09:56:32.752373 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:32.752308    2571 scope.go:117] "RemoveContainer" containerID="45b13026f61efa8c0d97f038a03db21c53dedeb3bc7127053628f603a660fdbf"
Jul 17 09:56:33.481519 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:33.481458    2571 generic.go:356] "Generic (PLEG): container finished" podID="f73f48ab-4405-44a0-a1be-09d07967f222" containerID="2b65b48c3b2b97aeeaee7d83ee710ead6918ffa63e73357ab0bac461602b3332" exitCode=6
Jul 17 09:56:33.481519 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:33.481501    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg" event={"ID":"f73f48ab-4405-44a0-a1be-09d07967f222","Type":"ContainerDied","Data":"2b65b48c3b2b97aeeaee7d83ee710ead6918ffa63e73357ab0bac461602b3332"}
Jul 17 09:56:33.481720 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:33.481563    2571 scope.go:117] "RemoveContainer" containerID="45b13026f61efa8c0d97f038a03db21c53dedeb3bc7127053628f603a660fdbf"
Jul 17 09:56:33.481912 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:33.481886    2571 scope.go:117] "RemoveContainer" containerID="2b65b48c3b2b97aeeaee7d83ee710ead6918ffa63e73357ab0bac461602b3332"
Jul 17 09:56:33.482112 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:56:33.482092    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg_odh-ai-gateway-infra(f73f48ab-4405-44a0-a1be-09d07967f222)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg" podUID="f73f48ab-4405-44a0-a1be-09d07967f222"
Jul 17 09:56:34.522527 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:34.522492    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"]
Jul 17 09:56:34.666182 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:34.666151    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"
Jul 17 09:56:34.786828 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:34.786736    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f73f48ab-4405-44a0-a1be-09d07967f222-maas-api-service-ca\") pod \"f73f48ab-4405-44a0-a1be-09d07967f222\" (UID: \"f73f48ab-4405-44a0-a1be-09d07967f222\") "
Jul 17 09:56:34.787119 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:34.787092    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f73f48ab-4405-44a0-a1be-09d07967f222-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f73f48ab-4405-44a0-a1be-09d07967f222" (UID: "f73f48ab-4405-44a0-a1be-09d07967f222"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:56:34.887418 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:34.887359    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f73f48ab-4405-44a0-a1be-09d07967f222-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:56:35.496546 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:35.496517    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"
Jul 17 09:56:35.496546 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:35.496537    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg" event={"ID":"f73f48ab-4405-44a0-a1be-09d07967f222","Type":"ContainerDied","Data":"ce11c4fd0b8a0a9b7fc017b682c576271d1bcc7f7c4b325ed7a5c9708a35caa3"}
Jul 17 09:56:35.496808 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:35.496606    2571 scope.go:117] "RemoveContainer" containerID="2b65b48c3b2b97aeeaee7d83ee710ead6918ffa63e73357ab0bac461602b3332"
Jul 17 09:56:35.526317 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:35.526279    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"]
Jul 17 09:56:35.530810 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:35.530776    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-2gkfg"]
Jul 17 09:56:35.758949 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:35.758858    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f73f48ab-4405-44a0-a1be-09d07967f222" path="/var/lib/kubelet/pods/f73f48ab-4405-44a0-a1be-09d07967f222/volumes"
Jul 17 09:56:53.766302 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.766263    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"]
Jul 17 09:56:53.766940 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.766918    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerName="revoke-keys"
Jul 17 09:56:53.767029 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.766942    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerName="revoke-keys"
Jul 17 09:56:53.767029 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.766980    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73f48ab-4405-44a0-a1be-09d07967f222" containerName="revoke-keys"
Jul 17 09:56:53.767029 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.766990    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73f48ab-4405-44a0-a1be-09d07967f222" containerName="revoke-keys"
Jul 17 09:56:53.767029 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767004    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerName="revoke-keys"
Jul 17 09:56:53.767029 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767013    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerName="revoke-keys"
Jul 17 09:56:53.767029 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767021    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73f48ab-4405-44a0-a1be-09d07967f222" containerName="revoke-keys"
Jul 17 09:56:53.767029 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767030    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73f48ab-4405-44a0-a1be-09d07967f222" containerName="revoke-keys"
Jul 17 09:56:53.767370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767040    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerName="revoke-keys"
Jul 17 09:56:53.767370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767048    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerName="revoke-keys"
Jul 17 09:56:53.767370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767071    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerName="revoke-keys"
Jul 17 09:56:53.767370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767079    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerName="revoke-keys"
Jul 17 09:56:53.767370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767185    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f73f48ab-4405-44a0-a1be-09d07967f222" containerName="revoke-keys"
Jul 17 09:56:53.767370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767198    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerName="revoke-keys"
Jul 17 09:56:53.767370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767209    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerName="revoke-keys"
Jul 17 09:56:53.767370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767218    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f73f48ab-4405-44a0-a1be-09d07967f222" containerName="revoke-keys"
Jul 17 09:56:53.767370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767227    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f73f48ab-4405-44a0-a1be-09d07967f222" containerName="revoke-keys"
Jul 17 09:56:53.767370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767236    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="52d8d6a9-0d96-4cb1-8f53-2dd5474a1832" containerName="revoke-keys"
Jul 17 09:56:53.767370 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.767246    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1662001-2d08-40d7-9a06-2e23e9f60145" containerName="revoke-keys"
Jul 17 09:56:53.770732 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.770711    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"
Jul 17 09:56:53.773713 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.773695    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:56:53.780015 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.779991    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"]
Jul 17 09:56:53.854293 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.854253    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e96d32d-dee8-4008-b5a1-7676c7814946-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-9256g\" (UID: \"7e96d32d-dee8-4008-b5a1-7676c7814946\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"
Jul 17 09:56:53.955184 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.955148    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e96d32d-dee8-4008-b5a1-7676c7814946-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-9256g\" (UID: \"7e96d32d-dee8-4008-b5a1-7676c7814946\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"
Jul 17 09:56:53.955922 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:53.955900    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e96d32d-dee8-4008-b5a1-7676c7814946-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-9256g\" (UID: \"7e96d32d-dee8-4008-b5a1-7676c7814946\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"
Jul 17 09:56:54.083911 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:54.083829    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"
Jul 17 09:56:54.225433 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:54.225407    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"]
Jul 17 09:56:54.228012 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:56:54.227984    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e96d32d_dee8_4008_b5a1_7676c7814946.slice/crio-520719f19870d8bf0566bae65c203b45213e19679da17d6ccf886983fd5c7dce WatchSource:0}: Error finding container 520719f19870d8bf0566bae65c203b45213e19679da17d6ccf886983fd5c7dce: Status 404 returned error can't find the container with id 520719f19870d8bf0566bae65c203b45213e19679da17d6ccf886983fd5c7dce
Jul 17 09:56:54.607648 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:54.607556    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g" event={"ID":"7e96d32d-dee8-4008-b5a1-7676c7814946","Type":"ContainerStarted","Data":"f54056828e3342a4bd2e9387e04b4b6de2debd73edc6810f941e58d497d989e6"}
Jul 17 09:56:54.607648 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:54.607635    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g" event={"ID":"7e96d32d-dee8-4008-b5a1-7676c7814946","Type":"ContainerStarted","Data":"520719f19870d8bf0566bae65c203b45213e19679da17d6ccf886983fd5c7dce"}
Jul 17 09:56:54.626436 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:54.626384    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g" podStartSLOduration=1.626369662 podStartE2EDuration="1.626369662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:56:54.625452141 +0000 UTC m=+2298.690272868" watchObservedRunningTime="2026-07-17 09:56:54.626369662 +0000 UTC m=+2298.691190383"
Jul 17 09:56:55.614612 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:55.614578    2571 generic.go:356] "Generic (PLEG): container finished" podID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerID="f54056828e3342a4bd2e9387e04b4b6de2debd73edc6810f941e58d497d989e6" exitCode=6
Jul 17 09:56:55.615052 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:55.614676    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g" event={"ID":"7e96d32d-dee8-4008-b5a1-7676c7814946","Type":"ContainerDied","Data":"f54056828e3342a4bd2e9387e04b4b6de2debd73edc6810f941e58d497d989e6"}
Jul 17 09:56:55.615052 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:55.614841    2571 scope.go:117] "RemoveContainer" containerID="f54056828e3342a4bd2e9387e04b4b6de2debd73edc6810f941e58d497d989e6"
Jul 17 09:56:56.622490 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:56.622374    2571 generic.go:356] "Generic (PLEG): container finished" podID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerID="c4f9785291ecabfbe02dfca12e4e5fb76feaf7f981c50ab525d9151eae4cc086" exitCode=6
Jul 17 09:56:56.622902 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:56.622455    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g" event={"ID":"7e96d32d-dee8-4008-b5a1-7676c7814946","Type":"ContainerDied","Data":"c4f9785291ecabfbe02dfca12e4e5fb76feaf7f981c50ab525d9151eae4cc086"}
Jul 17 09:56:56.622902 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:56.622538    2571 scope.go:117] "RemoveContainer" containerID="f54056828e3342a4bd2e9387e04b4b6de2debd73edc6810f941e58d497d989e6"
Jul 17 09:56:56.622902 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:56.622659    2571 scope.go:117] "RemoveContainer" containerID="c4f9785291ecabfbe02dfca12e4e5fb76feaf7f981c50ab525d9151eae4cc086"
Jul 17 09:56:56.623016 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:56:56.622937    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-9256g_odh-ai-gateway-infra(7e96d32d-dee8-4008-b5a1-7676c7814946)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946"
Jul 17 09:56:57.629914 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:56:57.629886    2571 scope.go:117] "RemoveContainer" containerID="c4f9785291ecabfbe02dfca12e4e5fb76feaf7f981c50ab525d9151eae4cc086"
Jul 17 09:56:57.630287 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:56:57.630096    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-9256g_odh-ai-gateway-infra(7e96d32d-dee8-4008-b5a1-7676c7814946)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946"
Jul 17 09:57:08.753108 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:08.753069    2571 scope.go:117] "RemoveContainer" containerID="c4f9785291ecabfbe02dfca12e4e5fb76feaf7f981c50ab525d9151eae4cc086"
Jul 17 09:57:09.698760 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:09.698725    2571 generic.go:356] "Generic (PLEG): container finished" podID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerID="b1f36b8c690ace8b5fc8adedf06a33768c8904b8b1ef2a9eba7ef88bcbe2f058" exitCode=6
Jul 17 09:57:09.698956 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:09.698798    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g" event={"ID":"7e96d32d-dee8-4008-b5a1-7676c7814946","Type":"ContainerDied","Data":"b1f36b8c690ace8b5fc8adedf06a33768c8904b8b1ef2a9eba7ef88bcbe2f058"}
Jul 17 09:57:09.698956 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:09.698860    2571 scope.go:117] "RemoveContainer" containerID="c4f9785291ecabfbe02dfca12e4e5fb76feaf7f981c50ab525d9151eae4cc086"
Jul 17 09:57:09.699260 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:09.699242    2571 scope.go:117] "RemoveContainer" containerID="b1f36b8c690ace8b5fc8adedf06a33768c8904b8b1ef2a9eba7ef88bcbe2f058"
Jul 17 09:57:09.699488 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:57:09.699448    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-9256g_odh-ai-gateway-infra(7e96d32d-dee8-4008-b5a1-7676c7814946)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946"
Jul 17 09:57:10.729737 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:10.729700    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"]
Jul 17 09:57:10.874584 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:10.874556    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"
Jul 17 09:57:11.018166 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:11.018077    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e96d32d-dee8-4008-b5a1-7676c7814946-maas-api-service-ca\") pod \"7e96d32d-dee8-4008-b5a1-7676c7814946\" (UID: \"7e96d32d-dee8-4008-b5a1-7676c7814946\") "
Jul 17 09:57:11.018434 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:11.018410    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e96d32d-dee8-4008-b5a1-7676c7814946-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7e96d32d-dee8-4008-b5a1-7676c7814946" (UID: "7e96d32d-dee8-4008-b5a1-7676c7814946"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:57:11.119351 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:11.119310    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e96d32d-dee8-4008-b5a1-7676c7814946-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:57:11.713407 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:11.713375    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"
Jul 17 09:57:11.713598 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:11.713416    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g" event={"ID":"7e96d32d-dee8-4008-b5a1-7676c7814946","Type":"ContainerDied","Data":"520719f19870d8bf0566bae65c203b45213e19679da17d6ccf886983fd5c7dce"}
Jul 17 09:57:11.713598 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:11.713521    2571 scope.go:117] "RemoveContainer" containerID="b1f36b8c690ace8b5fc8adedf06a33768c8904b8b1ef2a9eba7ef88bcbe2f058"
Jul 17 09:57:11.737236 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:11.737201    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"]
Jul 17 09:57:11.742811 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:11.742780    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9256g"]
Jul 17 09:57:11.759385 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:11.759341    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946" path="/var/lib/kubelet/pods/7e96d32d-dee8-4008-b5a1-7676c7814946/volumes"
Jul 17 09:57:15.099068 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099025    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"]
Jul 17 09:57:15.099692 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099673    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerName="revoke-keys"
Jul 17 09:57:15.099692 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099694    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerName="revoke-keys"
Jul 17 09:57:15.099847 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099704    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerName="revoke-keys"
Jul 17 09:57:15.099847 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099711    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerName="revoke-keys"
Jul 17 09:57:15.099847 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099736    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerName="revoke-keys"
Jul 17 09:57:15.099847 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099744    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerName="revoke-keys"
Jul 17 09:57:15.099847 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099773    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73f48ab-4405-44a0-a1be-09d07967f222" containerName="revoke-keys"
Jul 17 09:57:15.099847 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099782    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73f48ab-4405-44a0-a1be-09d07967f222" containerName="revoke-keys"
Jul 17 09:57:15.100155 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099884    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerName="revoke-keys"
Jul 17 09:57:15.100155 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099899    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerName="revoke-keys"
Jul 17 09:57:15.100155 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.099911    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e96d32d-dee8-4008-b5a1-7676c7814946" containerName="revoke-keys"
Jul 17 09:57:15.104552 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.104519    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"
Jul 17 09:57:15.107617 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.107595    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:57:15.111163 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.111136    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"]
Jul 17 09:57:15.257192 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.257150    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4\" (UID: \"1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"
Jul 17 09:57:15.357985 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.357903    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4\" (UID: \"1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"
Jul 17 09:57:15.358532 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.358511    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4\" (UID: \"1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"
Jul 17 09:57:15.417408 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.417357    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"
Jul 17 09:57:15.769831 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:15.769780    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"]
Jul 17 09:57:15.776005 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:57:15.775975    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c31ea46_b3a8_42d1_983e_a5b9c6ee8b38.slice/crio-ddd9d5901ec1b0f569e6c4e919c9c3943eec6cca1f8d348415df7f0becf21c2a WatchSource:0}: Error finding container ddd9d5901ec1b0f569e6c4e919c9c3943eec6cca1f8d348415df7f0becf21c2a: Status 404 returned error can't find the container with id ddd9d5901ec1b0f569e6c4e919c9c3943eec6cca1f8d348415df7f0becf21c2a
Jul 17 09:57:16.743702 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:16.743667    2571 generic.go:356] "Generic (PLEG): container finished" podID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerID="095c263ecdf068c342fb44d2c0f166c8b513ae743b8bc29e634371c842a61690" exitCode=6
Jul 17 09:57:16.744080 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:16.743715    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4" event={"ID":"1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38","Type":"ContainerDied","Data":"095c263ecdf068c342fb44d2c0f166c8b513ae743b8bc29e634371c842a61690"}
Jul 17 09:57:16.744080 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:16.743765    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4" event={"ID":"1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38","Type":"ContainerStarted","Data":"ddd9d5901ec1b0f569e6c4e919c9c3943eec6cca1f8d348415df7f0becf21c2a"}
Jul 17 09:57:16.744080 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:16.743969    2571 scope.go:117] "RemoveContainer" containerID="095c263ecdf068c342fb44d2c0f166c8b513ae743b8bc29e634371c842a61690"
Jul 17 09:57:17.751394 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:17.751356    2571 generic.go:356] "Generic (PLEG): container finished" podID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerID="dd80a1def20f360722145980c1dce29317b56fa89bb7ba69207d370199f16546" exitCode=6
Jul 17 09:57:17.751887 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:17.751415    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4" event={"ID":"1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38","Type":"ContainerDied","Data":"dd80a1def20f360722145980c1dce29317b56fa89bb7ba69207d370199f16546"}
Jul 17 09:57:17.751887 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:17.751476    2571 scope.go:117] "RemoveContainer" containerID="095c263ecdf068c342fb44d2c0f166c8b513ae743b8bc29e634371c842a61690"
Jul 17 09:57:17.751887 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:17.751729    2571 scope.go:117] "RemoveContainer" containerID="dd80a1def20f360722145980c1dce29317b56fa89bb7ba69207d370199f16546"
Jul 17 09:57:17.752049 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:57:17.751987    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4_odh-ai-gateway-infra(1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38"
Jul 17 09:57:18.759170 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:18.759140    2571 scope.go:117] "RemoveContainer" containerID="dd80a1def20f360722145980c1dce29317b56fa89bb7ba69207d370199f16546"
Jul 17 09:57:18.759564 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:57:18.759369    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4_odh-ai-gateway-infra(1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38"
Jul 17 09:57:31.752170 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:31.752091    2571 scope.go:117] "RemoveContainer" containerID="dd80a1def20f360722145980c1dce29317b56fa89bb7ba69207d370199f16546"
Jul 17 09:57:32.845867 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:32.845830    2571 generic.go:356] "Generic (PLEG): container finished" podID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerID="adf7f221bea82e28b7877ab493955214f3fe73c88a1be49db50e498900db4427" exitCode=6
Jul 17 09:57:32.846305 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:32.845899    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4" event={"ID":"1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38","Type":"ContainerDied","Data":"adf7f221bea82e28b7877ab493955214f3fe73c88a1be49db50e498900db4427"}
Jul 17 09:57:32.846305 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:32.845951    2571 scope.go:117] "RemoveContainer" containerID="dd80a1def20f360722145980c1dce29317b56fa89bb7ba69207d370199f16546"
Jul 17 09:57:32.846305 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:32.846239    2571 scope.go:117] "RemoveContainer" containerID="adf7f221bea82e28b7877ab493955214f3fe73c88a1be49db50e498900db4427"
Jul 17 09:57:32.846532 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:57:32.846502    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4_odh-ai-gateway-infra(1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38"
Jul 17 09:57:33.876116 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:33.876072    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"]
Jul 17 09:57:34.019010 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:34.018985    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"
Jul 17 09:57:34.030512 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:34.030456    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38-maas-api-service-ca\") pod \"1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38\" (UID: \"1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38\") "
Jul 17 09:57:34.030789 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:34.030765    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" (UID: "1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:57:34.131102 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:34.131013    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:57:34.860404 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:34.860374    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"
Jul 17 09:57:34.860621 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:34.860409    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4" event={"ID":"1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38","Type":"ContainerDied","Data":"ddd9d5901ec1b0f569e6c4e919c9c3943eec6cca1f8d348415df7f0becf21c2a"}
Jul 17 09:57:34.860621 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:34.860483    2571 scope.go:117] "RemoveContainer" containerID="adf7f221bea82e28b7877ab493955214f3fe73c88a1be49db50e498900db4427"
Jul 17 09:57:34.884113 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:34.884083    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"]
Jul 17 09:57:34.889425 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:34.889400    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-6dlm4"]
Jul 17 09:57:35.759174 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:35.759127    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" path="/var/lib/kubelet/pods/1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38/volumes"
Jul 17 09:57:43.809187 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.809132    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"]
Jul 17 09:57:43.809649 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.809594    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerName="revoke-keys"
Jul 17 09:57:43.809649 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.809607    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerName="revoke-keys"
Jul 17 09:57:43.809649 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.809615    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerName="revoke-keys"
Jul 17 09:57:43.809649 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.809621    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerName="revoke-keys"
Jul 17 09:57:43.809649 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.809641    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerName="revoke-keys"
Jul 17 09:57:43.809649 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.809647    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerName="revoke-keys"
Jul 17 09:57:43.809845 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.809708    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerName="revoke-keys"
Jul 17 09:57:43.809845 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.809717    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerName="revoke-keys"
Jul 17 09:57:43.809845 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.809725    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c31ea46-b3a8-42d1-983e-a5b9c6ee8b38" containerName="revoke-keys"
Jul 17 09:57:43.813033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.813007    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"
Jul 17 09:57:43.816087 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.816063    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:57:43.822046 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.822012    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"]
Jul 17 09:57:43.914091 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:43.914054    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee35000c-3f00-406d-8fbb-8808bc0aeed2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m\" (UID: \"ee35000c-3f00-406d-8fbb-8808bc0aeed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"
Jul 17 09:57:44.015576 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:44.015533    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee35000c-3f00-406d-8fbb-8808bc0aeed2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m\" (UID: \"ee35000c-3f00-406d-8fbb-8808bc0aeed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"
Jul 17 09:57:44.016241 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:44.016216    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee35000c-3f00-406d-8fbb-8808bc0aeed2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m\" (UID: \"ee35000c-3f00-406d-8fbb-8808bc0aeed2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"
Jul 17 09:57:44.126183 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:44.126090    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"
Jul 17 09:57:44.277146 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:44.277115    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"]
Jul 17 09:57:44.278601 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:57:44.278571    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee35000c_3f00_406d_8fbb_8808bc0aeed2.slice/crio-9b892e6225c4cfb3bc510fcce13f91869f71ea15b3f7eb1b61a7b899a7ac631c WatchSource:0}: Error finding container 9b892e6225c4cfb3bc510fcce13f91869f71ea15b3f7eb1b61a7b899a7ac631c: Status 404 returned error can't find the container with id 9b892e6225c4cfb3bc510fcce13f91869f71ea15b3f7eb1b61a7b899a7ac631c
Jul 17 09:57:44.920968 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:44.920931    2571 generic.go:356] "Generic (PLEG): container finished" podID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerID="548d50aa53c4b077ea422c9f5198c08998378381cac40f871fcd83c342f6b949" exitCode=6
Jul 17 09:57:44.921363 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:44.921023    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m" event={"ID":"ee35000c-3f00-406d-8fbb-8808bc0aeed2","Type":"ContainerDied","Data":"548d50aa53c4b077ea422c9f5198c08998378381cac40f871fcd83c342f6b949"}
Jul 17 09:57:44.921363 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:44.921079    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m" event={"ID":"ee35000c-3f00-406d-8fbb-8808bc0aeed2","Type":"ContainerStarted","Data":"9b892e6225c4cfb3bc510fcce13f91869f71ea15b3f7eb1b61a7b899a7ac631c"}
Jul 17 09:57:44.921363 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:44.921206    2571 scope.go:117] "RemoveContainer" containerID="548d50aa53c4b077ea422c9f5198c08998378381cac40f871fcd83c342f6b949"
Jul 17 09:57:45.929485 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:45.929375    2571 generic.go:356] "Generic (PLEG): container finished" podID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerID="c9c6491d0d8750eb6188a28904d8db114f246173b6ff587fbbfdf438b5c99901" exitCode=6
Jul 17 09:57:45.929913 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:45.929483    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m" event={"ID":"ee35000c-3f00-406d-8fbb-8808bc0aeed2","Type":"ContainerDied","Data":"c9c6491d0d8750eb6188a28904d8db114f246173b6ff587fbbfdf438b5c99901"}
Jul 17 09:57:45.929913 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:45.929551    2571 scope.go:117] "RemoveContainer" containerID="548d50aa53c4b077ea422c9f5198c08998378381cac40f871fcd83c342f6b949"
Jul 17 09:57:45.929913 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:45.929665    2571 scope.go:117] "RemoveContainer" containerID="c9c6491d0d8750eb6188a28904d8db114f246173b6ff587fbbfdf438b5c99901"
Jul 17 09:57:45.930081 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:57:45.929924    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m_odh-ai-gateway-infra(ee35000c-3f00-406d-8fbb-8808bc0aeed2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2"
Jul 17 09:57:46.938242 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:46.938214    2571 scope.go:117] "RemoveContainer" containerID="c9c6491d0d8750eb6188a28904d8db114f246173b6ff587fbbfdf438b5c99901"
Jul 17 09:57:46.938682 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:57:46.938496    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m_odh-ai-gateway-infra(ee35000c-3f00-406d-8fbb-8808bc0aeed2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2"
Jul 17 09:57:57.755901 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:57.755864    2571 scope.go:117] "RemoveContainer" containerID="c9c6491d0d8750eb6188a28904d8db114f246173b6ff587fbbfdf438b5c99901"
Jul 17 09:57:59.006554 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:59.006516    2571 generic.go:356] "Generic (PLEG): container finished" podID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerID="361f20f6d1d1f0e97f73109cd88faf3ce3d94e82d875784f6b857bc4ec23bd10" exitCode=6
Jul 17 09:57:59.006938 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:59.006587    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m" event={"ID":"ee35000c-3f00-406d-8fbb-8808bc0aeed2","Type":"ContainerDied","Data":"361f20f6d1d1f0e97f73109cd88faf3ce3d94e82d875784f6b857bc4ec23bd10"}
Jul 17 09:57:59.006938 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:59.006675    2571 scope.go:117] "RemoveContainer" containerID="c9c6491d0d8750eb6188a28904d8db114f246173b6ff587fbbfdf438b5c99901"
Jul 17 09:57:59.007083 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:57:59.006972    2571 scope.go:117] "RemoveContainer" containerID="361f20f6d1d1f0e97f73109cd88faf3ce3d94e82d875784f6b857bc4ec23bd10"
Jul 17 09:57:59.007222 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:57:59.007199    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m_odh-ai-gateway-infra(ee35000c-3f00-406d-8fbb-8808bc0aeed2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2"
Jul 17 09:58:00.040641 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:00.040606    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"]
Jul 17 09:58:00.186707 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:00.186682    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"
Jul 17 09:58:00.274724 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:00.274682    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee35000c-3f00-406d-8fbb-8808bc0aeed2-maas-api-service-ca\") pod \"ee35000c-3f00-406d-8fbb-8808bc0aeed2\" (UID: \"ee35000c-3f00-406d-8fbb-8808bc0aeed2\") "
Jul 17 09:58:00.275071 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:00.275048    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee35000c-3f00-406d-8fbb-8808bc0aeed2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ee35000c-3f00-406d-8fbb-8808bc0aeed2" (UID: "ee35000c-3f00-406d-8fbb-8808bc0aeed2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:58:00.375856 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:00.375748    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee35000c-3f00-406d-8fbb-8808bc0aeed2-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:58:01.021043 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:01.021009    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"
Jul 17 09:58:01.021043 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:01.021039    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m" event={"ID":"ee35000c-3f00-406d-8fbb-8808bc0aeed2","Type":"ContainerDied","Data":"9b892e6225c4cfb3bc510fcce13f91869f71ea15b3f7eb1b61a7b899a7ac631c"}
Jul 17 09:58:01.021290 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:01.021101    2571 scope.go:117] "RemoveContainer" containerID="361f20f6d1d1f0e97f73109cd88faf3ce3d94e82d875784f6b857bc4ec23bd10"
Jul 17 09:58:01.047224 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:01.047197    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"]
Jul 17 09:58:01.049774 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:01.049752    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-lc56m"]
Jul 17 09:58:01.758413 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:01.758381    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" path="/var/lib/kubelet/pods/ee35000c-3f00-406d-8fbb-8808bc0aeed2/volumes"
Jul 17 09:58:33.853408 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.853368    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"]
Jul 17 09:58:33.853903 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.853831    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerName="revoke-keys"
Jul 17 09:58:33.853903 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.853844    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerName="revoke-keys"
Jul 17 09:58:33.853903 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.853860    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerName="revoke-keys"
Jul 17 09:58:33.853903 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.853866    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerName="revoke-keys"
Jul 17 09:58:33.854038 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.853925    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerName="revoke-keys"
Jul 17 09:58:33.854038 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.853936    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerName="revoke-keys"
Jul 17 09:58:33.856942 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.856925    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"
Jul 17 09:58:33.859652 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.859632    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:58:33.866764 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.866640    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"]
Jul 17 09:58:33.869220 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.869195    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e315af7-2e71-4938-8b4b-588e00df03bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b\" (UID: \"6e315af7-2e71-4938-8b4b-588e00df03bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"
Jul 17 09:58:33.969986 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.969945    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e315af7-2e71-4938-8b4b-588e00df03bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b\" (UID: \"6e315af7-2e71-4938-8b4b-588e00df03bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"
Jul 17 09:58:33.970665 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:33.970642    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e315af7-2e71-4938-8b4b-588e00df03bb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b\" (UID: \"6e315af7-2e71-4938-8b4b-588e00df03bb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"
Jul 17 09:58:34.168221 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:34.168127    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"
Jul 17 09:58:34.535897 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:34.535868    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"]
Jul 17 09:58:34.537675 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:58:34.537649    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e315af7_2e71_4938_8b4b_588e00df03bb.slice/crio-1fca1a01127d7a9f79a427a80d8ab81f3f77c71719c27b5f33f10caff5dd2f29 WatchSource:0}: Error finding container 1fca1a01127d7a9f79a427a80d8ab81f3f77c71719c27b5f33f10caff5dd2f29: Status 404 returned error can't find the container with id 1fca1a01127d7a9f79a427a80d8ab81f3f77c71719c27b5f33f10caff5dd2f29
Jul 17 09:58:34.843929 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:34.843834    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"]
Jul 17 09:58:34.844381 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:34.844361    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerName="revoke-keys"
Jul 17 09:58:34.844510 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:34.844384    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerName="revoke-keys"
Jul 17 09:58:34.844587 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:34.844512    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee35000c-3f00-406d-8fbb-8808bc0aeed2" containerName="revoke-keys"
Jul 17 09:58:34.847744 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:34.847722    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"
Jul 17 09:58:34.855962 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:34.855387    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"]
Jul 17 09:58:34.877358 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:34.877317    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7e7b15c-33a5-4ed5-909c-230d77eb6e82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr\" (UID: \"e7e7b15c-33a5-4ed5-909c-230d77eb6e82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"
Jul 17 09:58:34.978394 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:34.978355    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7e7b15c-33a5-4ed5-909c-230d77eb6e82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr\" (UID: \"e7e7b15c-33a5-4ed5-909c-230d77eb6e82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"
Jul 17 09:58:34.979052 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:34.979032    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7e7b15c-33a5-4ed5-909c-230d77eb6e82-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr\" (UID: \"e7e7b15c-33a5-4ed5-909c-230d77eb6e82\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"
Jul 17 09:58:35.160139 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:35.160109    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"
Jul 17 09:58:35.223326 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:35.223294    2571 generic.go:356] "Generic (PLEG): container finished" podID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerID="7c00738c6db7e7b7a38efe43481aa4ad4ed6d56edd91639ce604000c5ed2d39a" exitCode=6
Jul 17 09:58:35.223498 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:35.223379    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b" event={"ID":"6e315af7-2e71-4938-8b4b-588e00df03bb","Type":"ContainerDied","Data":"7c00738c6db7e7b7a38efe43481aa4ad4ed6d56edd91639ce604000c5ed2d39a"}
Jul 17 09:58:35.223498 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:35.223444    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b" event={"ID":"6e315af7-2e71-4938-8b4b-588e00df03bb","Type":"ContainerStarted","Data":"1fca1a01127d7a9f79a427a80d8ab81f3f77c71719c27b5f33f10caff5dd2f29"}
Jul 17 09:58:35.223740 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:35.223725    2571 scope.go:117] "RemoveContainer" containerID="7c00738c6db7e7b7a38efe43481aa4ad4ed6d56edd91639ce604000c5ed2d39a"
Jul 17 09:58:35.301495 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:35.301454    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"]
Jul 17 09:58:35.303276 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:58:35.303246    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7e7b15c_33a5_4ed5_909c_230d77eb6e82.slice/crio-cffe8dd675fe312c553705053691c06ed51ffea821482d6fbcb0e3b93eea9f78 WatchSource:0}: Error finding container cffe8dd675fe312c553705053691c06ed51ffea821482d6fbcb0e3b93eea9f78: Status 404 returned error can't find the container with id cffe8dd675fe312c553705053691c06ed51ffea821482d6fbcb0e3b93eea9f78
Jul 17 09:58:35.857028 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:58:35.856997    2571 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e315af7_2e71_4938_8b4b_588e00df03bb.slice/crio-913def79a52e4ccbcf684749cc532312bd2924d1deef66007dda4ddddf24acc4.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e315af7_2e71_4938_8b4b_588e00df03bb.slice/crio-conmon-913def79a52e4ccbcf684749cc532312bd2924d1deef66007dda4ddddf24acc4.scope\": RecentStats: unable to find data in memory cache]"
Jul 17 09:58:36.231166 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:36.231135    2571 generic.go:356] "Generic (PLEG): container finished" podID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerID="913def79a52e4ccbcf684749cc532312bd2924d1deef66007dda4ddddf24acc4" exitCode=6
Jul 17 09:58:36.231346 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:36.231207    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b" event={"ID":"6e315af7-2e71-4938-8b4b-588e00df03bb","Type":"ContainerDied","Data":"913def79a52e4ccbcf684749cc532312bd2924d1deef66007dda4ddddf24acc4"}
Jul 17 09:58:36.231346 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:36.231261    2571 scope.go:117] "RemoveContainer" containerID="7c00738c6db7e7b7a38efe43481aa4ad4ed6d56edd91639ce604000c5ed2d39a"
Jul 17 09:58:36.231496 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:36.231457    2571 scope.go:117] "RemoveContainer" containerID="913def79a52e4ccbcf684749cc532312bd2924d1deef66007dda4ddddf24acc4"
Jul 17 09:58:36.231753 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:58:36.231722    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b_odh-ai-gateway-infra(6e315af7-2e71-4938-8b4b-588e00df03bb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb"
Jul 17 09:58:36.233048 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:36.233025    2571 generic.go:356] "Generic (PLEG): container finished" podID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerID="7e9eaaa0e6166416e24d9bf69bf9ab5e44db0a566d94de01db0e75001e867391" exitCode=6
Jul 17 09:58:36.233136 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:36.233050    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr" event={"ID":"e7e7b15c-33a5-4ed5-909c-230d77eb6e82","Type":"ContainerDied","Data":"7e9eaaa0e6166416e24d9bf69bf9ab5e44db0a566d94de01db0e75001e867391"}
Jul 17 09:58:36.233136 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:36.233101    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr" event={"ID":"e7e7b15c-33a5-4ed5-909c-230d77eb6e82","Type":"ContainerStarted","Data":"cffe8dd675fe312c553705053691c06ed51ffea821482d6fbcb0e3b93eea9f78"}
Jul 17 09:58:36.233257 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:36.233245    2571 scope.go:117] "RemoveContainer" containerID="7e9eaaa0e6166416e24d9bf69bf9ab5e44db0a566d94de01db0e75001e867391"
Jul 17 09:58:37.240698 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:37.240661    2571 scope.go:117] "RemoveContainer" containerID="913def79a52e4ccbcf684749cc532312bd2924d1deef66007dda4ddddf24acc4"
Jul 17 09:58:37.241129 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:58:37.240892    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b_odh-ai-gateway-infra(6e315af7-2e71-4938-8b4b-588e00df03bb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb"
Jul 17 09:58:37.242303 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:37.242278    2571 generic.go:356] "Generic (PLEG): container finished" podID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerID="a4e14dabf0db45590164bf8c09ba96fc70579b91ecb21310c4ee68e549ce5394" exitCode=6
Jul 17 09:58:37.242410 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:37.242352    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr" event={"ID":"e7e7b15c-33a5-4ed5-909c-230d77eb6e82","Type":"ContainerDied","Data":"a4e14dabf0db45590164bf8c09ba96fc70579b91ecb21310c4ee68e549ce5394"}
Jul 17 09:58:37.242454 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:37.242441    2571 scope.go:117] "RemoveContainer" containerID="7e9eaaa0e6166416e24d9bf69bf9ab5e44db0a566d94de01db0e75001e867391"
Jul 17 09:58:37.242553 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:37.242533    2571 scope.go:117] "RemoveContainer" containerID="a4e14dabf0db45590164bf8c09ba96fc70579b91ecb21310c4ee68e549ce5394"
Jul 17 09:58:37.242757 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:58:37.242734    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr_odh-ai-gateway-infra(e7e7b15c-33a5-4ed5-909c-230d77eb6e82)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82"
Jul 17 09:58:37.553028 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:37.552943    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:58:37.561480 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:37.561441    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 09:58:38.249539 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:38.249509    2571 scope.go:117] "RemoveContainer" containerID="a4e14dabf0db45590164bf8c09ba96fc70579b91ecb21310c4ee68e549ce5394"
Jul 17 09:58:38.249916 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:58:38.249723    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr_odh-ai-gateway-infra(e7e7b15c-33a5-4ed5-909c-230d77eb6e82)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82"
Jul 17 09:58:48.752853 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:48.752819    2571 scope.go:117] "RemoveContainer" containerID="913def79a52e4ccbcf684749cc532312bd2924d1deef66007dda4ddddf24acc4"
Jul 17 09:58:49.322756 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:49.322660    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b" event={"ID":"6e315af7-2e71-4938-8b4b-588e00df03bb","Type":"ContainerStarted","Data":"0aad07a26a9e56334676f9380a9d32cd5e477d17a5475ac348b65d0c0735a8f3"}
Jul 17 09:58:49.344947 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:49.344893    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b" podStartSLOduration=16.344873242 podStartE2EDuration="16.344873242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:58:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:58:49.341804128 +0000 UTC m=+2413.406624852" watchObservedRunningTime="2026-07-17 09:58:49.344873242 +0000 UTC m=+2413.409693965"
Jul 17 09:58:49.760485 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:49.760432    2571 scope.go:117] "RemoveContainer" containerID="a4e14dabf0db45590164bf8c09ba96fc70579b91ecb21310c4ee68e549ce5394"
Jul 17 09:58:50.330324 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:50.330239    2571 generic.go:356] "Generic (PLEG): container finished" podID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerID="0aad07a26a9e56334676f9380a9d32cd5e477d17a5475ac348b65d0c0735a8f3" exitCode=6
Jul 17 09:58:50.330506 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:50.330313    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b" event={"ID":"6e315af7-2e71-4938-8b4b-588e00df03bb","Type":"ContainerDied","Data":"0aad07a26a9e56334676f9380a9d32cd5e477d17a5475ac348b65d0c0735a8f3"}
Jul 17 09:58:50.330506 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:50.330381    2571 scope.go:117] "RemoveContainer" containerID="913def79a52e4ccbcf684749cc532312bd2924d1deef66007dda4ddddf24acc4"
Jul 17 09:58:50.330723 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:50.330700    2571 scope.go:117] "RemoveContainer" containerID="0aad07a26a9e56334676f9380a9d32cd5e477d17a5475ac348b65d0c0735a8f3"
Jul 17 09:58:50.331010 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:58:50.330984    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b_odh-ai-gateway-infra(6e315af7-2e71-4938-8b4b-588e00df03bb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb"
Jul 17 09:58:50.332489 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:50.332445    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr" event={"ID":"e7e7b15c-33a5-4ed5-909c-230d77eb6e82","Type":"ContainerStarted","Data":"45a836eae4f3a1e58aae9c5c1036d698485b29ba8a78ff79a5770907315d3da5"}
Jul 17 09:58:50.359272 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:50.359246    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"]
Jul 17 09:58:50.370033 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:50.369991    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr" podStartSLOduration=16.369976396 podStartE2EDuration="16.369976396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:58:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:58:50.369402059 +0000 UTC m=+2414.434222782" watchObservedRunningTime="2026-07-17 09:58:50.369976396 +0000 UTC m=+2414.434797118"
Jul 17 09:58:51.341550 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:51.341513    2571 generic.go:356] "Generic (PLEG): container finished" podID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerID="45a836eae4f3a1e58aae9c5c1036d698485b29ba8a78ff79a5770907315d3da5" exitCode=6
Jul 17 09:58:51.341865 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:51.341591    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr" event={"ID":"e7e7b15c-33a5-4ed5-909c-230d77eb6e82","Type":"ContainerDied","Data":"45a836eae4f3a1e58aae9c5c1036d698485b29ba8a78ff79a5770907315d3da5"}
Jul 17 09:58:51.341865 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:51.341669    2571 scope.go:117] "RemoveContainer" containerID="a4e14dabf0db45590164bf8c09ba96fc70579b91ecb21310c4ee68e549ce5394"
Jul 17 09:58:51.341990 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:51.341974    2571 scope.go:117] "RemoveContainer" containerID="45a836eae4f3a1e58aae9c5c1036d698485b29ba8a78ff79a5770907315d3da5"
Jul 17 09:58:51.342186 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:58:51.342168    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr_odh-ai-gateway-infra(e7e7b15c-33a5-4ed5-909c-230d77eb6e82)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82"
Jul 17 09:58:51.387867 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:51.387831    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"]
Jul 17 09:58:51.490259 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:51.490233    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"
Jul 17 09:58:51.528061 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:51.528028    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e315af7-2e71-4938-8b4b-588e00df03bb-maas-api-service-ca\") pod \"6e315af7-2e71-4938-8b4b-588e00df03bb\" (UID: \"6e315af7-2e71-4938-8b4b-588e00df03bb\") "
Jul 17 09:58:51.528372 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:51.528348    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e315af7-2e71-4938-8b4b-588e00df03bb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6e315af7-2e71-4938-8b4b-588e00df03bb" (UID: "6e315af7-2e71-4938-8b4b-588e00df03bb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:58:51.629710 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:51.629623    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e315af7-2e71-4938-8b4b-588e00df03bb-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:58:52.349486 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:52.349429    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"
Jul 17 09:58:52.349486 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:52.349428    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b" event={"ID":"6e315af7-2e71-4938-8b4b-588e00df03bb","Type":"ContainerDied","Data":"1fca1a01127d7a9f79a427a80d8ab81f3f77c71719c27b5f33f10caff5dd2f29"}
Jul 17 09:58:52.349984 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:52.349548    2571 scope.go:117] "RemoveContainer" containerID="0aad07a26a9e56334676f9380a9d32cd5e477d17a5475ac348b65d0c0735a8f3"
Jul 17 09:58:52.370402 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:52.370379    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"]
Jul 17 09:58:52.376108 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:52.376082    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-bvx4b"]
Jul 17 09:58:52.496250 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:52.496225    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"
Jul 17 09:58:52.538697 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:52.538668    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7e7b15c-33a5-4ed5-909c-230d77eb6e82-maas-api-service-ca\") pod \"e7e7b15c-33a5-4ed5-909c-230d77eb6e82\" (UID: \"e7e7b15c-33a5-4ed5-909c-230d77eb6e82\") "
Jul 17 09:58:52.539022 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:52.538999    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7e7b15c-33a5-4ed5-909c-230d77eb6e82-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e7e7b15c-33a5-4ed5-909c-230d77eb6e82" (UID: "e7e7b15c-33a5-4ed5-909c-230d77eb6e82"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:58:52.640182 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:52.640107    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7e7b15c-33a5-4ed5-909c-230d77eb6e82-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:58:53.359172 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:53.359142    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"
Jul 17 09:58:53.359172 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:53.359173    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr" event={"ID":"e7e7b15c-33a5-4ed5-909c-230d77eb6e82","Type":"ContainerDied","Data":"cffe8dd675fe312c553705053691c06ed51ffea821482d6fbcb0e3b93eea9f78"}
Jul 17 09:58:53.359826 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:53.359229    2571 scope.go:117] "RemoveContainer" containerID="45a836eae4f3a1e58aae9c5c1036d698485b29ba8a78ff79a5770907315d3da5"
Jul 17 09:58:53.383297 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:53.383275    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"]
Jul 17 09:58:53.388139 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:53.388118    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-vcztr"]
Jul 17 09:58:53.758780 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:53.758750    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb" path="/var/lib/kubelet/pods/6e315af7-2e71-4938-8b4b-588e00df03bb/volumes"
Jul 17 09:58:53.759096 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:58:53.759083    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" path="/var/lib/kubelet/pods/e7e7b15c-33a5-4ed5-909c-230d77eb6e82/volumes"
Jul 17 09:59:23.912500 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.912440    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"]
Jul 17 09:59:23.913320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913105    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerName="revoke-keys"
Jul 17 09:59:23.913320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913128    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerName="revoke-keys"
Jul 17 09:59:23.913320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913140    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerName="revoke-keys"
Jul 17 09:59:23.913320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913148    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerName="revoke-keys"
Jul 17 09:59:23.913320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913176    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerName="revoke-keys"
Jul 17 09:59:23.913320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913185    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerName="revoke-keys"
Jul 17 09:59:23.913320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913195    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerName="revoke-keys"
Jul 17 09:59:23.913320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913203    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerName="revoke-keys"
Jul 17 09:59:23.913320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913216    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerName="revoke-keys"
Jul 17 09:59:23.913320 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913223    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerName="revoke-keys"
Jul 17 09:59:23.913886 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913679    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerName="revoke-keys"
Jul 17 09:59:23.913886 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913698    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerName="revoke-keys"
Jul 17 09:59:23.913886 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913710    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerName="revoke-keys"
Jul 17 09:59:23.913886 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913721    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerName="revoke-keys"
Jul 17 09:59:23.913886 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.913732    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e315af7-2e71-4938-8b4b-588e00df03bb" containerName="revoke-keys"
Jul 17 09:59:23.917214 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.917190    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"]
Jul 17 09:59:23.917338 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.917295    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"
Jul 17 09:59:23.920760 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.920739    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 09:59:23.928124 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:23.928095    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5712920-c1b5-4393-bc20-80177921b51a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p\" (UID: \"e5712920-c1b5-4393-bc20-80177921b51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"
Jul 17 09:59:24.029147 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:24.029104    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5712920-c1b5-4393-bc20-80177921b51a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p\" (UID: \"e5712920-c1b5-4393-bc20-80177921b51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"
Jul 17 09:59:24.029814 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:24.029792    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5712920-c1b5-4393-bc20-80177921b51a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p\" (UID: \"e5712920-c1b5-4393-bc20-80177921b51a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"
Jul 17 09:59:24.228160 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:24.228120    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"
Jul 17 09:59:24.374439 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:24.374405    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"]
Jul 17 09:59:24.376088 ip-10-0-138-208 kubenswrapper[2571]: W0717 09:59:24.376061    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5712920_c1b5_4393_bc20_80177921b51a.slice/crio-5529d63c7e9b7b99436789c65387ec439ddfa0e0a6d38e21ce48b56bc0437034 WatchSource:0}: Error finding container 5529d63c7e9b7b99436789c65387ec439ddfa0e0a6d38e21ce48b56bc0437034: Status 404 returned error can't find the container with id 5529d63c7e9b7b99436789c65387ec439ddfa0e0a6d38e21ce48b56bc0437034
Jul 17 09:59:24.537737 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:24.537657    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p" event={"ID":"e5712920-c1b5-4393-bc20-80177921b51a","Type":"ContainerStarted","Data":"5529d63c7e9b7b99436789c65387ec439ddfa0e0a6d38e21ce48b56bc0437034"}
Jul 17 09:59:25.545554 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:25.545518    2571 generic.go:356] "Generic (PLEG): container finished" podID="e5712920-c1b5-4393-bc20-80177921b51a" containerID="0e9f40d5295f372ccc42072acf003940915d9f8547e8f6894efedd1481296c6f" exitCode=6
Jul 17 09:59:25.545946 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:25.545589    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p" event={"ID":"e5712920-c1b5-4393-bc20-80177921b51a","Type":"ContainerDied","Data":"0e9f40d5295f372ccc42072acf003940915d9f8547e8f6894efedd1481296c6f"}
Jul 17 09:59:25.545946 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:25.545794    2571 scope.go:117] "RemoveContainer" containerID="0e9f40d5295f372ccc42072acf003940915d9f8547e8f6894efedd1481296c6f"
Jul 17 09:59:26.553389 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:26.553300    2571 generic.go:356] "Generic (PLEG): container finished" podID="e5712920-c1b5-4393-bc20-80177921b51a" containerID="c537b81215b02bc9f1c7a533ab686c2a2f00803d7a7dbe1e27debb39dc6e0c3d" exitCode=6
Jul 17 09:59:26.553857 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:26.553388    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p" event={"ID":"e5712920-c1b5-4393-bc20-80177921b51a","Type":"ContainerDied","Data":"c537b81215b02bc9f1c7a533ab686c2a2f00803d7a7dbe1e27debb39dc6e0c3d"}
Jul 17 09:59:26.553857 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:26.553449    2571 scope.go:117] "RemoveContainer" containerID="0e9f40d5295f372ccc42072acf003940915d9f8547e8f6894efedd1481296c6f"
Jul 17 09:59:26.553857 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:26.553602    2571 scope.go:117] "RemoveContainer" containerID="c537b81215b02bc9f1c7a533ab686c2a2f00803d7a7dbe1e27debb39dc6e0c3d"
Jul 17 09:59:26.553857 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:59:26.553806    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p_odh-ai-gateway-infra(e5712920-c1b5-4393-bc20-80177921b51a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p" podUID="e5712920-c1b5-4393-bc20-80177921b51a"
Jul 17 09:59:27.561480 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:27.561438    2571 scope.go:117] "RemoveContainer" containerID="c537b81215b02bc9f1c7a533ab686c2a2f00803d7a7dbe1e27debb39dc6e0c3d"
Jul 17 09:59:27.561868 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:59:27.561648    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p_odh-ai-gateway-infra(e5712920-c1b5-4393-bc20-80177921b51a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p" podUID="e5712920-c1b5-4393-bc20-80177921b51a"
Jul 17 09:59:37.757010 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:37.756988    2571 scope.go:117] "RemoveContainer" containerID="c537b81215b02bc9f1c7a533ab686c2a2f00803d7a7dbe1e27debb39dc6e0c3d"
Jul 17 09:59:38.632453 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:38.632420    2571 generic.go:356] "Generic (PLEG): container finished" podID="e5712920-c1b5-4393-bc20-80177921b51a" containerID="599cdfe2239c82d20582b2ad022c58dc3e92756ee78e9c94ca751df13a132e6b" exitCode=6
Jul 17 09:59:38.632625 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:38.632506    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p" event={"ID":"e5712920-c1b5-4393-bc20-80177921b51a","Type":"ContainerDied","Data":"599cdfe2239c82d20582b2ad022c58dc3e92756ee78e9c94ca751df13a132e6b"}
Jul 17 09:59:38.632625 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:38.632572    2571 scope.go:117] "RemoveContainer" containerID="c537b81215b02bc9f1c7a533ab686c2a2f00803d7a7dbe1e27debb39dc6e0c3d"
Jul 17 09:59:38.632880 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:38.632860    2571 scope.go:117] "RemoveContainer" containerID="599cdfe2239c82d20582b2ad022c58dc3e92756ee78e9c94ca751df13a132e6b"
Jul 17 09:59:38.633138 ip-10-0-138-208 kubenswrapper[2571]: E0717 09:59:38.633111    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p_odh-ai-gateway-infra(e5712920-c1b5-4393-bc20-80177921b51a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p" podUID="e5712920-c1b5-4393-bc20-80177921b51a"
Jul 17 09:59:39.659423 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:39.659387    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"]
Jul 17 09:59:39.804709 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:39.804678    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"
Jul 17 09:59:39.869803 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:39.869765    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5712920-c1b5-4393-bc20-80177921b51a-maas-api-service-ca\") pod \"e5712920-c1b5-4393-bc20-80177921b51a\" (UID: \"e5712920-c1b5-4393-bc20-80177921b51a\") "
Jul 17 09:59:39.870138 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:39.870114    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5712920-c1b5-4393-bc20-80177921b51a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e5712920-c1b5-4393-bc20-80177921b51a" (UID: "e5712920-c1b5-4393-bc20-80177921b51a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:59:39.971270 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:39.971234    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5712920-c1b5-4393-bc20-80177921b51a-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 09:59:40.646884 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:40.646857    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"
Jul 17 09:59:40.646884 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:40.646875    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p" event={"ID":"e5712920-c1b5-4393-bc20-80177921b51a","Type":"ContainerDied","Data":"5529d63c7e9b7b99436789c65387ec439ddfa0e0a6d38e21ce48b56bc0437034"}
Jul 17 09:59:40.647164 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:40.646935    2571 scope.go:117] "RemoveContainer" containerID="599cdfe2239c82d20582b2ad022c58dc3e92756ee78e9c94ca751df13a132e6b"
Jul 17 09:59:40.705052 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:40.705015    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"]
Jul 17 09:59:40.712670 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:40.712639    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tfb5p"]
Jul 17 09:59:41.758385 ip-10-0-138-208 kubenswrapper[2571]: I0717 09:59:41.758342    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5712920-c1b5-4393-bc20-80177921b51a" path="/var/lib/kubelet/pods/e5712920-c1b5-4393-bc20-80177921b51a/volumes"
Jul 17 10:00:14.968084 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968043    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"]
Jul 17 10:00:14.970511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968496    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5712920-c1b5-4393-bc20-80177921b51a" containerName="revoke-keys"
Jul 17 10:00:14.970511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968508    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5712920-c1b5-4393-bc20-80177921b51a" containerName="revoke-keys"
Jul 17 10:00:14.970511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968520    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerName="revoke-keys"
Jul 17 10:00:14.970511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968525    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerName="revoke-keys"
Jul 17 10:00:14.970511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968533    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5712920-c1b5-4393-bc20-80177921b51a" containerName="revoke-keys"
Jul 17 10:00:14.970511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968540    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5712920-c1b5-4393-bc20-80177921b51a" containerName="revoke-keys"
Jul 17 10:00:14.970511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968545    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5712920-c1b5-4393-bc20-80177921b51a" containerName="revoke-keys"
Jul 17 10:00:14.970511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968551    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5712920-c1b5-4393-bc20-80177921b51a" containerName="revoke-keys"
Jul 17 10:00:14.970511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968619    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5712920-c1b5-4393-bc20-80177921b51a" containerName="revoke-keys"
Jul 17 10:00:14.970511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968630    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5712920-c1b5-4393-bc20-80177921b51a" containerName="revoke-keys"
Jul 17 10:00:14.970511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.968635    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7e7b15c-33a5-4ed5-909c-230d77eb6e82" containerName="revoke-keys"
Jul 17 10:00:14.971805 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.971771    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"
Jul 17 10:00:14.974603 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.974574    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:00:14.983231 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.983203    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"]
Jul 17 10:00:14.989943 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:14.989918    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23390974-80f5-4037-a165-80506f322104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks\" (UID: \"23390974-80f5-4037-a165-80506f322104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"
Jul 17 10:00:15.090741 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:15.090701    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23390974-80f5-4037-a165-80506f322104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks\" (UID: \"23390974-80f5-4037-a165-80506f322104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"
Jul 17 10:00:15.091345 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:15.091326    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23390974-80f5-4037-a165-80506f322104-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks\" (UID: \"23390974-80f5-4037-a165-80506f322104\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"
Jul 17 10:00:15.283201 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:15.283109    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"
Jul 17 10:00:15.428879 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:15.428846    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"]
Jul 17 10:00:15.430055 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:00:15.430027    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23390974_80f5_4037_a165_80506f322104.slice/crio-bb1bce0905f9b3c420c84b98e8dcad44849bf626efc7874e44f625a039f8e630 WatchSource:0}: Error finding container bb1bce0905f9b3c420c84b98e8dcad44849bf626efc7874e44f625a039f8e630: Status 404 returned error can't find the container with id bb1bce0905f9b3c420c84b98e8dcad44849bf626efc7874e44f625a039f8e630
Jul 17 10:00:15.842070 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:15.842031    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks" event={"ID":"23390974-80f5-4037-a165-80506f322104","Type":"ContainerStarted","Data":"8624b774e4f417ea22f961605b3525e4b078a7281b3d4477e7a78e33bf473f06"}
Jul 17 10:00:15.842222 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:15.842093    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks" event={"ID":"23390974-80f5-4037-a165-80506f322104","Type":"ContainerStarted","Data":"bb1bce0905f9b3c420c84b98e8dcad44849bf626efc7874e44f625a039f8e630"}
Jul 17 10:00:15.878850 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:15.878788    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks" podStartSLOduration=1.878773767 podStartE2EDuration="1.878773767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:00:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:00:15.876260585 +0000 UTC m=+2499.941081307" watchObservedRunningTime="2026-07-17 10:00:15.878773767 +0000 UTC m=+2499.943594487"
Jul 17 10:00:16.849106 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:16.849070    2571 generic.go:356] "Generic (PLEG): container finished" podID="23390974-80f5-4037-a165-80506f322104" containerID="8624b774e4f417ea22f961605b3525e4b078a7281b3d4477e7a78e33bf473f06" exitCode=6
Jul 17 10:00:16.849509 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:16.849158    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks" event={"ID":"23390974-80f5-4037-a165-80506f322104","Type":"ContainerDied","Data":"8624b774e4f417ea22f961605b3525e4b078a7281b3d4477e7a78e33bf473f06"}
Jul 17 10:00:16.849509 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:16.849312    2571 scope.go:117] "RemoveContainer" containerID="8624b774e4f417ea22f961605b3525e4b078a7281b3d4477e7a78e33bf473f06"
Jul 17 10:00:17.857259 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:17.857222    2571 generic.go:356] "Generic (PLEG): container finished" podID="23390974-80f5-4037-a165-80506f322104" containerID="1f1f47d379bd6ccbb134dc5c867215ee93c1070e06fe61ef6f86e17027638116" exitCode=6
Jul 17 10:00:17.857759 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:17.857291    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks" event={"ID":"23390974-80f5-4037-a165-80506f322104","Type":"ContainerDied","Data":"1f1f47d379bd6ccbb134dc5c867215ee93c1070e06fe61ef6f86e17027638116"}
Jul 17 10:00:17.857759 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:17.857339    2571 scope.go:117] "RemoveContainer" containerID="8624b774e4f417ea22f961605b3525e4b078a7281b3d4477e7a78e33bf473f06"
Jul 17 10:00:17.857759 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:17.857608    2571 scope.go:117] "RemoveContainer" containerID="1f1f47d379bd6ccbb134dc5c867215ee93c1070e06fe61ef6f86e17027638116"
Jul 17 10:00:17.857916 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:00:17.857870    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks_odh-ai-gateway-infra(23390974-80f5-4037-a165-80506f322104)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks" podUID="23390974-80f5-4037-a165-80506f322104"
Jul 17 10:00:18.865002 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:18.864977    2571 scope.go:117] "RemoveContainer" containerID="1f1f47d379bd6ccbb134dc5c867215ee93c1070e06fe61ef6f86e17027638116"
Jul 17 10:00:18.865484 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:00:18.865250    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks_odh-ai-gateway-infra(23390974-80f5-4037-a165-80506f322104)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks" podUID="23390974-80f5-4037-a165-80506f322104"
Jul 17 10:00:32.753026 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:32.752945    2571 scope.go:117] "RemoveContainer" containerID="1f1f47d379bd6ccbb134dc5c867215ee93c1070e06fe61ef6f86e17027638116"
Jul 17 10:00:33.955005 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:33.954968    2571 generic.go:356] "Generic (PLEG): container finished" podID="23390974-80f5-4037-a165-80506f322104" containerID="78caf1afa5d9dc2cfbf47a89e9617c9a7c34b60c224e22c8fc0804cf9c702e47" exitCode=6
Jul 17 10:00:33.955492 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:33.955041    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks" event={"ID":"23390974-80f5-4037-a165-80506f322104","Type":"ContainerDied","Data":"78caf1afa5d9dc2cfbf47a89e9617c9a7c34b60c224e22c8fc0804cf9c702e47"}
Jul 17 10:00:33.955492 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:33.955121    2571 scope.go:117] "RemoveContainer" containerID="1f1f47d379bd6ccbb134dc5c867215ee93c1070e06fe61ef6f86e17027638116"
Jul 17 10:00:33.955610 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:33.955529    2571 scope.go:117] "RemoveContainer" containerID="78caf1afa5d9dc2cfbf47a89e9617c9a7c34b60c224e22c8fc0804cf9c702e47"
Jul 17 10:00:33.955761 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:00:33.955732    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks_odh-ai-gateway-infra(23390974-80f5-4037-a165-80506f322104)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks" podUID="23390974-80f5-4037-a165-80506f322104"
Jul 17 10:00:34.989349 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:34.989309    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"]
Jul 17 10:00:35.129516 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:35.129488    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"
Jul 17 10:00:35.186611 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:35.186577    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23390974-80f5-4037-a165-80506f322104-maas-api-service-ca\") pod \"23390974-80f5-4037-a165-80506f322104\" (UID: \"23390974-80f5-4037-a165-80506f322104\") "
Jul 17 10:00:35.186966 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:35.186943    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23390974-80f5-4037-a165-80506f322104-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "23390974-80f5-4037-a165-80506f322104" (UID: "23390974-80f5-4037-a165-80506f322104"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:00:35.287957 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:35.287877    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23390974-80f5-4037-a165-80506f322104-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:00:35.970857 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:35.970818    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks" event={"ID":"23390974-80f5-4037-a165-80506f322104","Type":"ContainerDied","Data":"bb1bce0905f9b3c420c84b98e8dcad44849bf626efc7874e44f625a039f8e630"}
Jul 17 10:00:35.971026 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:35.970884    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"
Jul 17 10:00:35.971026 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:35.970886    2571 scope.go:117] "RemoveContainer" containerID="78caf1afa5d9dc2cfbf47a89e9617c9a7c34b60c224e22c8fc0804cf9c702e47"
Jul 17 10:00:35.993920 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:35.993883    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"]
Jul 17 10:00:36.003416 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:36.003387    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xgnks"]
Jul 17 10:00:37.759269 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:37.759239    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23390974-80f5-4037-a165-80506f322104" path="/var/lib/kubelet/pods/23390974-80f5-4037-a165-80506f322104/volumes"
Jul 17 10:00:45.343126 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.343087    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"]
Jul 17 10:00:45.343647 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.343545    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23390974-80f5-4037-a165-80506f322104" containerName="revoke-keys"
Jul 17 10:00:45.343647 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.343558    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="23390974-80f5-4037-a165-80506f322104" containerName="revoke-keys"
Jul 17 10:00:45.343647 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.343566    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23390974-80f5-4037-a165-80506f322104" containerName="revoke-keys"
Jul 17 10:00:45.343647 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.343572    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="23390974-80f5-4037-a165-80506f322104" containerName="revoke-keys"
Jul 17 10:00:45.343647 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.343582    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23390974-80f5-4037-a165-80506f322104" containerName="revoke-keys"
Jul 17 10:00:45.343647 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.343588    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="23390974-80f5-4037-a165-80506f322104" containerName="revoke-keys"
Jul 17 10:00:45.343647 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.343647    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="23390974-80f5-4037-a165-80506f322104" containerName="revoke-keys"
Jul 17 10:00:45.343892 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.343655    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5712920-c1b5-4393-bc20-80177921b51a" containerName="revoke-keys"
Jul 17 10:00:45.343892 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.343663    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="23390974-80f5-4037-a165-80506f322104" containerName="revoke-keys"
Jul 17 10:00:45.346736 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.346719    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"
Jul 17 10:00:45.350919 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.350896    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:00:45.361109 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.361079    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"]
Jul 17 10:00:45.483899 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.483861    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74339d2e-594a-43d4-b56f-d2fae8a9861d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv\" (UID: \"74339d2e-594a-43d4-b56f-d2fae8a9861d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"
Jul 17 10:00:45.584954 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.584915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74339d2e-594a-43d4-b56f-d2fae8a9861d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv\" (UID: \"74339d2e-594a-43d4-b56f-d2fae8a9861d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"
Jul 17 10:00:45.585581 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.585557    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74339d2e-594a-43d4-b56f-d2fae8a9861d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv\" (UID: \"74339d2e-594a-43d4-b56f-d2fae8a9861d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"
Jul 17 10:00:45.657007 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.656894    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"
Jul 17 10:00:45.809599 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:45.809573    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"]
Jul 17 10:00:45.811262 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:00:45.811230    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74339d2e_594a_43d4_b56f_d2fae8a9861d.slice/crio-518508839e687d9537091b784e156f6737822e1fa18cd547bd043067d90495b5 WatchSource:0}: Error finding container 518508839e687d9537091b784e156f6737822e1fa18cd547bd043067d90495b5: Status 404 returned error can't find the container with id 518508839e687d9537091b784e156f6737822e1fa18cd547bd043067d90495b5
Jul 17 10:00:46.029579 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:46.029545    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv" event={"ID":"74339d2e-594a-43d4-b56f-d2fae8a9861d","Type":"ContainerStarted","Data":"518508839e687d9537091b784e156f6737822e1fa18cd547bd043067d90495b5"}
Jul 17 10:00:47.037571 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:47.037532    2571 generic.go:356] "Generic (PLEG): container finished" podID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerID="ac6062581fd9a545852be5d4cea97ccbfeab33566fd89f9b332f3867b03b4c00" exitCode=6
Jul 17 10:00:47.038024 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:47.037622    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv" event={"ID":"74339d2e-594a-43d4-b56f-d2fae8a9861d","Type":"ContainerDied","Data":"ac6062581fd9a545852be5d4cea97ccbfeab33566fd89f9b332f3867b03b4c00"}
Jul 17 10:00:47.038024 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:47.037819    2571 scope.go:117] "RemoveContainer" containerID="ac6062581fd9a545852be5d4cea97ccbfeab33566fd89f9b332f3867b03b4c00"
Jul 17 10:00:48.045284 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:48.045251    2571 generic.go:356] "Generic (PLEG): container finished" podID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerID="57771425787a2c58652f172e98b651f87fd2ca5725656cb6c41b7b3842d73ebd" exitCode=6
Jul 17 10:00:48.045699 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:48.045336    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv" event={"ID":"74339d2e-594a-43d4-b56f-d2fae8a9861d","Type":"ContainerDied","Data":"57771425787a2c58652f172e98b651f87fd2ca5725656cb6c41b7b3842d73ebd"}
Jul 17 10:00:48.045699 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:48.045391    2571 scope.go:117] "RemoveContainer" containerID="ac6062581fd9a545852be5d4cea97ccbfeab33566fd89f9b332f3867b03b4c00"
Jul 17 10:00:48.045699 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:48.045632    2571 scope.go:117] "RemoveContainer" containerID="57771425787a2c58652f172e98b651f87fd2ca5725656cb6c41b7b3842d73ebd"
Jul 17 10:00:48.045871 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:00:48.045847    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv_odh-ai-gateway-infra(74339d2e-594a-43d4-b56f-d2fae8a9861d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d"
Jul 17 10:00:49.052699 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:00:49.052666    2571 scope.go:117] "RemoveContainer" containerID="57771425787a2c58652f172e98b651f87fd2ca5725656cb6c41b7b3842d73ebd"
Jul 17 10:00:49.053101 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:00:49.052872    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv_odh-ai-gateway-infra(74339d2e-594a-43d4-b56f-d2fae8a9861d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d"
Jul 17 10:01:00.752029 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:00.751998    2571 scope.go:117] "RemoveContainer" containerID="57771425787a2c58652f172e98b651f87fd2ca5725656cb6c41b7b3842d73ebd"
Jul 17 10:01:01.122341 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:01.122251    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv" event={"ID":"74339d2e-594a-43d4-b56f-d2fae8a9861d","Type":"ContainerStarted","Data":"8d9f3a4172cb30beb5e3d890b8ef7b3db4c65ceb309c7e1dd58b2e6fd3ed5317"}
Jul 17 10:01:02.129321 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:02.129285    2571 generic.go:356] "Generic (PLEG): container finished" podID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerID="8d9f3a4172cb30beb5e3d890b8ef7b3db4c65ceb309c7e1dd58b2e6fd3ed5317" exitCode=6
Jul 17 10:01:02.129769 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:02.129357    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv" event={"ID":"74339d2e-594a-43d4-b56f-d2fae8a9861d","Type":"ContainerDied","Data":"8d9f3a4172cb30beb5e3d890b8ef7b3db4c65ceb309c7e1dd58b2e6fd3ed5317"}
Jul 17 10:01:02.129769 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:02.129415    2571 scope.go:117] "RemoveContainer" containerID="57771425787a2c58652f172e98b651f87fd2ca5725656cb6c41b7b3842d73ebd"
Jul 17 10:01:02.129769 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:02.129701    2571 scope.go:117] "RemoveContainer" containerID="8d9f3a4172cb30beb5e3d890b8ef7b3db4c65ceb309c7e1dd58b2e6fd3ed5317"
Jul 17 10:01:02.129926 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:01:02.129901    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv_odh-ai-gateway-infra(74339d2e-594a-43d4-b56f-d2fae8a9861d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d"
Jul 17 10:01:02.195113 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:02.195080    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"]
Jul 17 10:01:03.279954 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:03.279926    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"
Jul 17 10:01:03.444331 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:03.444289    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74339d2e-594a-43d4-b56f-d2fae8a9861d-maas-api-service-ca\") pod \"74339d2e-594a-43d4-b56f-d2fae8a9861d\" (UID: \"74339d2e-594a-43d4-b56f-d2fae8a9861d\") "
Jul 17 10:01:03.444732 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:03.444710    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74339d2e-594a-43d4-b56f-d2fae8a9861d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "74339d2e-594a-43d4-b56f-d2fae8a9861d" (UID: "74339d2e-594a-43d4-b56f-d2fae8a9861d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:01:03.545210 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:03.545176    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74339d2e-594a-43d4-b56f-d2fae8a9861d-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:01:04.146210 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:04.146121    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"
Jul 17 10:01:04.146384 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:04.146118    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv" event={"ID":"74339d2e-594a-43d4-b56f-d2fae8a9861d","Type":"ContainerDied","Data":"518508839e687d9537091b784e156f6737822e1fa18cd547bd043067d90495b5"}
Jul 17 10:01:04.146384 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:04.146262    2571 scope.go:117] "RemoveContainer" containerID="8d9f3a4172cb30beb5e3d890b8ef7b3db4c65ceb309c7e1dd58b2e6fd3ed5317"
Jul 17 10:01:04.168971 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:04.168942    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"]
Jul 17 10:01:04.176763 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:04.176733    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9bttv"]
Jul 17 10:01:05.758494 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:05.758447    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d" path="/var/lib/kubelet/pods/74339d2e-594a-43d4-b56f-d2fae8a9861d/volumes"
Jul 17 10:01:15.046457 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.046421    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"]
Jul 17 10:01:15.046878 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.046863    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerName="revoke-keys"
Jul 17 10:01:15.046923 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.046880    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerName="revoke-keys"
Jul 17 10:01:15.046923 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.046899    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerName="revoke-keys"
Jul 17 10:01:15.046923 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.046905    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerName="revoke-keys"
Jul 17 10:01:15.047025 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.046970    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerName="revoke-keys"
Jul 17 10:01:15.047025 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.046978    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="23390974-80f5-4037-a165-80506f322104" containerName="revoke-keys"
Jul 17 10:01:15.047025 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.046987    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerName="revoke-keys"
Jul 17 10:01:15.047025 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.046994    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerName="revoke-keys"
Jul 17 10:01:15.049970 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.049952    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"
Jul 17 10:01:15.054316 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.054292    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:01:15.062081 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.062048    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"]
Jul 17 10:01:15.140940 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.140900    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6\" (UID: \"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"
Jul 17 10:01:15.242060 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.242022    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6\" (UID: \"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"
Jul 17 10:01:15.242682 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.242664    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6\" (UID: \"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"
Jul 17 10:01:15.361621 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.361528    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"
Jul 17 10:01:15.512655 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:15.512629    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"]
Jul 17 10:01:15.515003 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:01:15.514971    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5cd33bc_b6c0_44a9_8b2c_55e3cf8f8a41.slice/crio-9b53fc5ca7e198e981da2ae1d1e533f5de0d94575de434775c847b59691b751d WatchSource:0}: Error finding container 9b53fc5ca7e198e981da2ae1d1e533f5de0d94575de434775c847b59691b751d: Status 404 returned error can't find the container with id 9b53fc5ca7e198e981da2ae1d1e533f5de0d94575de434775c847b59691b751d
Jul 17 10:01:16.215726 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:16.215689    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6" event={"ID":"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41","Type":"ContainerStarted","Data":"8ed4e1106bc98aea2a1f72d2806a465b1091d5fc5f0bf61d66586db36012cf88"}
Jul 17 10:01:16.216126 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:16.215760    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6" event={"ID":"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41","Type":"ContainerStarted","Data":"9b53fc5ca7e198e981da2ae1d1e533f5de0d94575de434775c847b59691b751d"}
Jul 17 10:01:16.251532 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:16.251483    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6" podStartSLOduration=1.251450064 podStartE2EDuration="1.251450064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:01:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:01:16.249748389 +0000 UTC m=+2560.314569110" watchObservedRunningTime="2026-07-17 10:01:16.251450064 +0000 UTC m=+2560.316270785"
Jul 17 10:01:17.222919 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:17.222874    2571 generic.go:356] "Generic (PLEG): container finished" podID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerID="8ed4e1106bc98aea2a1f72d2806a465b1091d5fc5f0bf61d66586db36012cf88" exitCode=6
Jul 17 10:01:17.223380 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:17.222956    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6" event={"ID":"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41","Type":"ContainerDied","Data":"8ed4e1106bc98aea2a1f72d2806a465b1091d5fc5f0bf61d66586db36012cf88"}
Jul 17 10:01:17.223380 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:17.223121    2571 scope.go:117] "RemoveContainer" containerID="8ed4e1106bc98aea2a1f72d2806a465b1091d5fc5f0bf61d66586db36012cf88"
Jul 17 10:01:18.230123 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:18.230084    2571 generic.go:356] "Generic (PLEG): container finished" podID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerID="52cfce367311a82c4fb3d19b6b56e74981699d1437ebd98cda0e64ec2efc4f63" exitCode=6
Jul 17 10:01:18.230571 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:18.230169    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6" event={"ID":"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41","Type":"ContainerDied","Data":"52cfce367311a82c4fb3d19b6b56e74981699d1437ebd98cda0e64ec2efc4f63"}
Jul 17 10:01:18.230571 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:18.230258    2571 scope.go:117] "RemoveContainer" containerID="8ed4e1106bc98aea2a1f72d2806a465b1091d5fc5f0bf61d66586db36012cf88"
Jul 17 10:01:18.230571 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:18.230372    2571 scope.go:117] "RemoveContainer" containerID="52cfce367311a82c4fb3d19b6b56e74981699d1437ebd98cda0e64ec2efc4f63"
Jul 17 10:01:18.230688 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:01:18.230647    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6_odh-ai-gateway-infra(e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41"
Jul 17 10:01:19.237382 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:19.237350    2571 scope.go:117] "RemoveContainer" containerID="52cfce367311a82c4fb3d19b6b56e74981699d1437ebd98cda0e64ec2efc4f63"
Jul 17 10:01:19.237787 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:01:19.237601    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6_odh-ai-gateway-infra(e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41"
Jul 17 10:01:31.752729 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:31.752660    2571 scope.go:117] "RemoveContainer" containerID="52cfce367311a82c4fb3d19b6b56e74981699d1437ebd98cda0e64ec2efc4f63"
Jul 17 10:01:32.314019 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:32.313922    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6" event={"ID":"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41","Type":"ContainerStarted","Data":"13de46380714636c6c65bbe6f6d6bdcc43a53f2257f60f8da71554e8dd2a8612"}
Jul 17 10:01:33.328475 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:33.328418    2571 generic.go:356] "Generic (PLEG): container finished" podID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerID="13de46380714636c6c65bbe6f6d6bdcc43a53f2257f60f8da71554e8dd2a8612" exitCode=6
Jul 17 10:01:33.328959 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:33.328484    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6" event={"ID":"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41","Type":"ContainerDied","Data":"13de46380714636c6c65bbe6f6d6bdcc43a53f2257f60f8da71554e8dd2a8612"}
Jul 17 10:01:33.328959 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:33.328589    2571 scope.go:117] "RemoveContainer" containerID="52cfce367311a82c4fb3d19b6b56e74981699d1437ebd98cda0e64ec2efc4f63"
Jul 17 10:01:33.328959 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:33.328838    2571 scope.go:117] "RemoveContainer" containerID="13de46380714636c6c65bbe6f6d6bdcc43a53f2257f60f8da71554e8dd2a8612"
Jul 17 10:01:33.329089 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:01:33.329060    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6_odh-ai-gateway-infra(e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41"
Jul 17 10:01:33.377726 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:33.377683    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"]
Jul 17 10:01:34.465330 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:34.465306    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"
Jul 17 10:01:34.538166 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:34.538130    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41-maas-api-service-ca\") pod \"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41\" (UID: \"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41\") "
Jul 17 10:01:34.538551 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:34.538528    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" (UID: "e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:01:34.639957 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:34.639871    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:01:35.342863 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.342837    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"
Jul 17 10:01:35.343039 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.342837    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6" event={"ID":"e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41","Type":"ContainerDied","Data":"9b53fc5ca7e198e981da2ae1d1e533f5de0d94575de434775c847b59691b751d"}
Jul 17 10:01:35.343039 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.342973    2571 scope.go:117] "RemoveContainer" containerID="13de46380714636c6c65bbe6f6d6bdcc43a53f2257f60f8da71554e8dd2a8612"
Jul 17 10:01:35.368240 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.368201    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"]
Jul 17 10:01:35.370820 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.370793    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"]
Jul 17 10:01:35.371262 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.371245    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerName="revoke-keys"
Jul 17 10:01:35.371348 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.371265    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerName="revoke-keys"
Jul 17 10:01:35.371348 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.371308    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerName="revoke-keys"
Jul 17 10:01:35.371348 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.371318    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="74339d2e-594a-43d4-b56f-d2fae8a9861d" containerName="revoke-keys"
Jul 17 10:01:35.371348 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.371336    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerName="revoke-keys"
Jul 17 10:01:35.371348 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.371345    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerName="revoke-keys"
Jul 17 10:01:35.371630 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.371445    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerName="revoke-keys"
Jul 17 10:01:35.371630 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.371483    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerName="revoke-keys"
Jul 17 10:01:35.375957 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.375937    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"
Jul 17 10:01:35.380821 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.380790    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:01:35.383120 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.382959    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fn4z6"]
Jul 17 10:01:35.393086 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.393054    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"]
Jul 17 10:01:35.446392 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.446354    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/590977ea-371d-4893-93fd-e3ef5f34c7d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft\" (UID: \"590977ea-371d-4893-93fd-e3ef5f34c7d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"
Jul 17 10:01:35.547088 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.547041    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/590977ea-371d-4893-93fd-e3ef5f34c7d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft\" (UID: \"590977ea-371d-4893-93fd-e3ef5f34c7d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"
Jul 17 10:01:35.547716 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.547692    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/590977ea-371d-4893-93fd-e3ef5f34c7d7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft\" (UID: \"590977ea-371d-4893-93fd-e3ef5f34c7d7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"
Jul 17 10:01:35.686761 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.686665    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"
Jul 17 10:01:35.759746 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.759708    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" path="/var/lib/kubelet/pods/e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41/volumes"
Jul 17 10:01:35.833404 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:35.833377    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"]
Jul 17 10:01:35.834792 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:01:35.834761    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590977ea_371d_4893_93fd_e3ef5f34c7d7.slice/crio-ff80852b9d2757116469eb274d6dc455c6b7ef31adb7d1f0d30001b32471a5b3 WatchSource:0}: Error finding container ff80852b9d2757116469eb274d6dc455c6b7ef31adb7d1f0d30001b32471a5b3: Status 404 returned error can't find the container with id ff80852b9d2757116469eb274d6dc455c6b7ef31adb7d1f0d30001b32471a5b3
Jul 17 10:01:36.350775 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:36.350683    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft" event={"ID":"590977ea-371d-4893-93fd-e3ef5f34c7d7","Type":"ContainerStarted","Data":"59f85d5b23d718ad8a917731bfbfb5c39e2295dfa881a50d28ae09d6fb35a25d"}
Jul 17 10:01:36.350775 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:36.350740    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft" event={"ID":"590977ea-371d-4893-93fd-e3ef5f34c7d7","Type":"ContainerStarted","Data":"ff80852b9d2757116469eb274d6dc455c6b7ef31adb7d1f0d30001b32471a5b3"}
Jul 17 10:01:36.370228 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:36.370177    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft" podStartSLOduration=1.370160389 podStartE2EDuration="1.370160389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:01:36.368657341 +0000 UTC m=+2580.433478064" watchObservedRunningTime="2026-07-17 10:01:36.370160389 +0000 UTC m=+2580.434981225"
Jul 17 10:01:37.358657 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:37.358621    2571 generic.go:356] "Generic (PLEG): container finished" podID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerID="59f85d5b23d718ad8a917731bfbfb5c39e2295dfa881a50d28ae09d6fb35a25d" exitCode=6
Jul 17 10:01:37.359133 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:37.358718    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft" event={"ID":"590977ea-371d-4893-93fd-e3ef5f34c7d7","Type":"ContainerDied","Data":"59f85d5b23d718ad8a917731bfbfb5c39e2295dfa881a50d28ae09d6fb35a25d"}
Jul 17 10:01:37.359133 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:37.358934    2571 scope.go:117] "RemoveContainer" containerID="59f85d5b23d718ad8a917731bfbfb5c39e2295dfa881a50d28ae09d6fb35a25d"
Jul 17 10:01:38.366161 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:38.366121    2571 generic.go:356] "Generic (PLEG): container finished" podID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerID="ae9f2020214eb8b8deb78cf87f5d0cf6e136eb0e839221c5a91bcd9d314144a8" exitCode=6
Jul 17 10:01:38.366571 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:38.366186    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft" event={"ID":"590977ea-371d-4893-93fd-e3ef5f34c7d7","Type":"ContainerDied","Data":"ae9f2020214eb8b8deb78cf87f5d0cf6e136eb0e839221c5a91bcd9d314144a8"}
Jul 17 10:01:38.366571 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:38.366245    2571 scope.go:117] "RemoveContainer" containerID="59f85d5b23d718ad8a917731bfbfb5c39e2295dfa881a50d28ae09d6fb35a25d"
Jul 17 10:01:38.366571 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:38.366402    2571 scope.go:117] "RemoveContainer" containerID="ae9f2020214eb8b8deb78cf87f5d0cf6e136eb0e839221c5a91bcd9d314144a8"
Jul 17 10:01:38.366723 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:01:38.366698    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft_odh-ai-gateway-infra(590977ea-371d-4893-93fd-e3ef5f34c7d7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7"
Jul 17 10:01:39.373720 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:39.373687    2571 scope.go:117] "RemoveContainer" containerID="ae9f2020214eb8b8deb78cf87f5d0cf6e136eb0e839221c5a91bcd9d314144a8"
Jul 17 10:01:39.374189 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:01:39.373938    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft_odh-ai-gateway-infra(590977ea-371d-4893-93fd-e3ef5f34c7d7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7"
Jul 17 10:01:53.752573 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:53.752540    2571 scope.go:117] "RemoveContainer" containerID="ae9f2020214eb8b8deb78cf87f5d0cf6e136eb0e839221c5a91bcd9d314144a8"
Jul 17 10:01:54.459689 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:54.459656    2571 generic.go:356] "Generic (PLEG): container finished" podID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerID="e7e0b51ca2474cc6a4c67a75d530e368a3635e0e13c79676bfa66ec8559fde2f" exitCode=6
Jul 17 10:01:54.459841 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:54.459721    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft" event={"ID":"590977ea-371d-4893-93fd-e3ef5f34c7d7","Type":"ContainerDied","Data":"e7e0b51ca2474cc6a4c67a75d530e368a3635e0e13c79676bfa66ec8559fde2f"}
Jul 17 10:01:54.459841 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:54.459792    2571 scope.go:117] "RemoveContainer" containerID="ae9f2020214eb8b8deb78cf87f5d0cf6e136eb0e839221c5a91bcd9d314144a8"
Jul 17 10:01:54.460101 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:54.460080    2571 scope.go:117] "RemoveContainer" containerID="e7e0b51ca2474cc6a4c67a75d530e368a3635e0e13c79676bfa66ec8559fde2f"
Jul 17 10:01:54.460323 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:01:54.460303    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft_odh-ai-gateway-infra(590977ea-371d-4893-93fd-e3ef5f34c7d7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7"
Jul 17 10:01:55.491346 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:55.491312    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"]
Jul 17 10:01:55.620957 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:55.620934    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"
Jul 17 10:01:55.627753 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:55.627721    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/590977ea-371d-4893-93fd-e3ef5f34c7d7-maas-api-service-ca\") pod \"590977ea-371d-4893-93fd-e3ef5f34c7d7\" (UID: \"590977ea-371d-4893-93fd-e3ef5f34c7d7\") "
Jul 17 10:01:55.628040 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:55.628021    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/590977ea-371d-4893-93fd-e3ef5f34c7d7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "590977ea-371d-4893-93fd-e3ef5f34c7d7" (UID: "590977ea-371d-4893-93fd-e3ef5f34c7d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:01:55.729129 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:55.729092    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/590977ea-371d-4893-93fd-e3ef5f34c7d7-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:01:56.473714 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:56.473685    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"
Jul 17 10:01:56.473925 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:56.473711    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft" event={"ID":"590977ea-371d-4893-93fd-e3ef5f34c7d7","Type":"ContainerDied","Data":"ff80852b9d2757116469eb274d6dc455c6b7ef31adb7d1f0d30001b32471a5b3"}
Jul 17 10:01:56.473925 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:56.473782    2571 scope.go:117] "RemoveContainer" containerID="e7e0b51ca2474cc6a4c67a75d530e368a3635e0e13c79676bfa66ec8559fde2f"
Jul 17 10:01:56.496796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:56.496765    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"]
Jul 17 10:01:56.501519 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:56.501492    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5b5ft"]
Jul 17 10:01:57.759204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:01:57.759172    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7" path="/var/lib/kubelet/pods/590977ea-371d-4893-93fd-e3ef5f34c7d7/volumes"
Jul 17 10:02:04.145602 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.145566    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"]
Jul 17 10:02:04.146033 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.146018    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerName="revoke-keys"
Jul 17 10:02:04.146077 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.146034    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerName="revoke-keys"
Jul 17 10:02:04.146077 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.146058    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerName="revoke-keys"
Jul 17 10:02:04.146077 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.146064    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerName="revoke-keys"
Jul 17 10:02:04.146077 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.146070    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerName="revoke-keys"
Jul 17 10:02:04.146077 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.146076    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerName="revoke-keys"
Jul 17 10:02:04.146232 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.146084    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerName="revoke-keys"
Jul 17 10:02:04.146232 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.146089    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerName="revoke-keys"
Jul 17 10:02:04.146232 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.146150    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerName="revoke-keys"
Jul 17 10:02:04.146232 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.146158    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5cd33bc-b6c0-44a9-8b2c-55e3cf8f8a41" containerName="revoke-keys"
Jul 17 10:02:04.146232 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.146166    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerName="revoke-keys"
Jul 17 10:02:04.150701 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.150684    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"
Jul 17 10:02:04.153573 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.153551    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:02:04.159079 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.158767    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"]
Jul 17 10:02:04.204624 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.204584    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1be80a2-0748-42b4-9699-746fa28f4b3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj\" (UID: \"d1be80a2-0748-42b4-9699-746fa28f4b3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"
Jul 17 10:02:04.305860 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.305821    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1be80a2-0748-42b4-9699-746fa28f4b3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj\" (UID: \"d1be80a2-0748-42b4-9699-746fa28f4b3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"
Jul 17 10:02:04.306488 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.306441    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1be80a2-0748-42b4-9699-746fa28f4b3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj\" (UID: \"d1be80a2-0748-42b4-9699-746fa28f4b3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"
Jul 17 10:02:04.463379 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.463329    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"
Jul 17 10:02:04.844689 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:04.844666    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"]
Jul 17 10:02:04.846637 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:02:04.846607    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1be80a2_0748_42b4_9699_746fa28f4b3b.slice/crio-cb3b94583afbb847e5df3891e892ecabb40b2e042116115e3186e88133ac326a WatchSource:0}: Error finding container cb3b94583afbb847e5df3891e892ecabb40b2e042116115e3186e88133ac326a: Status 404 returned error can't find the container with id cb3b94583afbb847e5df3891e892ecabb40b2e042116115e3186e88133ac326a
Jul 17 10:02:05.062921 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.062883    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd"]
Jul 17 10:02:05.063389 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.063376    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="590977ea-371d-4893-93fd-e3ef5f34c7d7" containerName="revoke-keys"
Jul 17 10:02:05.066825 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.066808    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd"
Jul 17 10:02:05.076884 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.076858    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd"]
Jul 17 10:02:05.112033 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.111945    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b12cee4-ac8a-4c89-84a7-a05bfe9528c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd\" (UID: \"1b12cee4-ac8a-4c89-84a7-a05bfe9528c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd"
Jul 17 10:02:05.213486 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.213431    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b12cee4-ac8a-4c89-84a7-a05bfe9528c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd\" (UID: \"1b12cee4-ac8a-4c89-84a7-a05bfe9528c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd"
Jul 17 10:02:05.214077 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.214058    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b12cee4-ac8a-4c89-84a7-a05bfe9528c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd\" (UID: \"1b12cee4-ac8a-4c89-84a7-a05bfe9528c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd"
Jul 17 10:02:05.378701 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.378612    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd"
Jul 17 10:02:05.527762 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.527720    2571 generic.go:356] "Generic (PLEG): container finished" podID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerID="58cfdc24af3d83e23533d6f5a3470572013709d96fd938983eb4a5d42c3ca75c" exitCode=6
Jul 17 10:02:05.527929 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.527813    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj" event={"ID":"d1be80a2-0748-42b4-9699-746fa28f4b3b","Type":"ContainerDied","Data":"58cfdc24af3d83e23533d6f5a3470572013709d96fd938983eb4a5d42c3ca75c"}
Jul 17 10:02:05.527929 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.527875    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj" event={"ID":"d1be80a2-0748-42b4-9699-746fa28f4b3b","Type":"ContainerStarted","Data":"cb3b94583afbb847e5df3891e892ecabb40b2e042116115e3186e88133ac326a"}
Jul 17 10:02:05.528026 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.527978    2571 scope.go:117] "RemoveContainer" containerID="58cfdc24af3d83e23533d6f5a3470572013709d96fd938983eb4a5d42c3ca75c"
Jul 17 10:02:05.536996 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:05.536971    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd"]
Jul 17 10:02:05.539726 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:02:05.539700    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b12cee4_ac8a_4c89_84a7_a05bfe9528c8.slice/crio-bebc3c141385d6b00edb51bdc0a2d4062cc12f925cfebec81946b73a5bcb6c3a WatchSource:0}: Error finding container bebc3c141385d6b00edb51bdc0a2d4062cc12f925cfebec81946b73a5bcb6c3a: Status 404 returned error can't find the container with id bebc3c141385d6b00edb51bdc0a2d4062cc12f925cfebec81946b73a5bcb6c3a
Jul 17 10:02:06.535045 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:06.535010    2571 generic.go:356] "Generic (PLEG): container finished" podID="1b12cee4-ac8a-4c89-84a7-a05bfe9528c8" containerID="557ee76b016f052304f76e247e51c657f6d04454b49680f032785f220a9e2718" exitCode=0
Jul 17 10:02:06.535513 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:06.535097    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd" event={"ID":"1b12cee4-ac8a-4c89-84a7-a05bfe9528c8","Type":"ContainerDied","Data":"557ee76b016f052304f76e247e51c657f6d04454b49680f032785f220a9e2718"}
Jul 17 10:02:06.535513 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:06.535161    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd" event={"ID":"1b12cee4-ac8a-4c89-84a7-a05bfe9528c8","Type":"ContainerStarted","Data":"bebc3c141385d6b00edb51bdc0a2d4062cc12f925cfebec81946b73a5bcb6c3a"}
Jul 17 10:02:06.537153 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:06.537128    2571 generic.go:356] "Generic (PLEG): container finished" podID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerID="64e9598a1cb9e55ffe6f70a6646c7f395d12ae381db53636350ba3b48edc0eb3" exitCode=6
Jul 17 10:02:06.537264 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:06.537189    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj" event={"ID":"d1be80a2-0748-42b4-9699-746fa28f4b3b","Type":"ContainerDied","Data":"64e9598a1cb9e55ffe6f70a6646c7f395d12ae381db53636350ba3b48edc0eb3"}
Jul 17 10:02:06.537264 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:06.537242    2571 scope.go:117] "RemoveContainer" containerID="58cfdc24af3d83e23533d6f5a3470572013709d96fd938983eb4a5d42c3ca75c"
Jul 17 10:02:06.537437 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:06.537418    2571 scope.go:117] "RemoveContainer" containerID="64e9598a1cb9e55ffe6f70a6646c7f395d12ae381db53636350ba3b48edc0eb3"
Jul 17 10:02:06.537749 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:06.537722    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj_odh-ai-gateway-infra(d1be80a2-0748-42b4-9699-746fa28f4b3b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b"
Jul 17 10:02:07.544568 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:07.544536    2571 scope.go:117] "RemoveContainer" containerID="64e9598a1cb9e55ffe6f70a6646c7f395d12ae381db53636350ba3b48edc0eb3"
Jul 17 10:02:07.544949 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:07.544735    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj_odh-ai-gateway-infra(d1be80a2-0748-42b4-9699-746fa28f4b3b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b"
Jul 17 10:02:07.688593 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:07.688568    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd"
Jul 17 10:02:07.739359 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:07.739323    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b12cee4-ac8a-4c89-84a7-a05bfe9528c8-maas-api-service-ca\") pod \"1b12cee4-ac8a-4c89-84a7-a05bfe9528c8\" (UID: \"1b12cee4-ac8a-4c89-84a7-a05bfe9528c8\") "
Jul 17 10:02:07.739739 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:07.739705    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b12cee4-ac8a-4c89-84a7-a05bfe9528c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1b12cee4-ac8a-4c89-84a7-a05bfe9528c8" (UID: "1b12cee4-ac8a-4c89-84a7-a05bfe9528c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:02:07.841193 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:07.841099    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b12cee4-ac8a-4c89-84a7-a05bfe9528c8-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:02:08.551293 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:08.551258    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd" event={"ID":"1b12cee4-ac8a-4c89-84a7-a05bfe9528c8","Type":"ContainerDied","Data":"bebc3c141385d6b00edb51bdc0a2d4062cc12f925cfebec81946b73a5bcb6c3a"}
Jul 17 10:02:08.551703 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:08.551309    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bebc3c141385d6b00edb51bdc0a2d4062cc12f925cfebec81946b73a5bcb6c3a"
Jul 17 10:02:08.551703 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:08.551309    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd"
Jul 17 10:02:15.140095 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:15.140053    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"]
Jul 17 10:02:15.140523 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:15.140505    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b12cee4-ac8a-4c89-84a7-a05bfe9528c8" containerName="revoke-keys"
Jul 17 10:02:15.140523 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:15.140518    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b12cee4-ac8a-4c89-84a7-a05bfe9528c8" containerName="revoke-keys"
Jul 17 10:02:15.140609 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:15.140604    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b12cee4-ac8a-4c89-84a7-a05bfe9528c8" containerName="revoke-keys"
Jul 17 10:02:15.143610 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:15.143590    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"
Jul 17 10:02:15.154975 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:15.154950    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"]
Jul 17 10:02:15.210321 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:15.210281    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a16d5f4-d5aa-428c-a2f6-07b786b6babe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m\" (UID: \"0a16d5f4-d5aa-428c-a2f6-07b786b6babe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"
Jul 17 10:02:15.311040 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:15.311000    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a16d5f4-d5aa-428c-a2f6-07b786b6babe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m\" (UID: \"0a16d5f4-d5aa-428c-a2f6-07b786b6babe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"
Jul 17 10:02:15.311695 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:15.311673    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a16d5f4-d5aa-428c-a2f6-07b786b6babe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m\" (UID: \"0a16d5f4-d5aa-428c-a2f6-07b786b6babe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"
Jul 17 10:02:15.455636 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:15.455597    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"
Jul 17 10:02:15.635291 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:15.635254    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"]
Jul 17 10:02:15.638316 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:02:15.638290    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a16d5f4_d5aa_428c_a2f6_07b786b6babe.slice/crio-97970d8ba42060548e191adf14caea041376ed308b4e245a192ecc1b6fbdf29e WatchSource:0}: Error finding container 97970d8ba42060548e191adf14caea041376ed308b4e245a192ecc1b6fbdf29e: Status 404 returned error can't find the container with id 97970d8ba42060548e191adf14caea041376ed308b4e245a192ecc1b6fbdf29e
Jul 17 10:02:16.601166 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:16.601070    2571 generic.go:356] "Generic (PLEG): container finished" podID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerID="f1883bbfa3468c9ab37c0290b027bbca32a10ba9e1b9c2d231ca14b993667924" exitCode=6
Jul 17 10:02:16.601166 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:16.601157    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m" event={"ID":"0a16d5f4-d5aa-428c-a2f6-07b786b6babe","Type":"ContainerDied","Data":"f1883bbfa3468c9ab37c0290b027bbca32a10ba9e1b9c2d231ca14b993667924"}
Jul 17 10:02:16.601694 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:16.601208    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m" event={"ID":"0a16d5f4-d5aa-428c-a2f6-07b786b6babe","Type":"ContainerStarted","Data":"97970d8ba42060548e191adf14caea041376ed308b4e245a192ecc1b6fbdf29e"}
Jul 17 10:02:16.601694 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:16.601397    2571 scope.go:117] "RemoveContainer" containerID="f1883bbfa3468c9ab37c0290b027bbca32a10ba9e1b9c2d231ca14b993667924"
Jul 17 10:02:17.609566 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:17.609441    2571 generic.go:356] "Generic (PLEG): container finished" podID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerID="61a59b65ff091ee819b25b8b0b09269f0f5d238be012346bae788fd9919d9be0" exitCode=6
Jul 17 10:02:17.609566 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:17.609513    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m" event={"ID":"0a16d5f4-d5aa-428c-a2f6-07b786b6babe","Type":"ContainerDied","Data":"61a59b65ff091ee819b25b8b0b09269f0f5d238be012346bae788fd9919d9be0"}
Jul 17 10:02:17.610036 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:17.609609    2571 scope.go:117] "RemoveContainer" containerID="f1883bbfa3468c9ab37c0290b027bbca32a10ba9e1b9c2d231ca14b993667924"
Jul 17 10:02:17.610036 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:17.609854    2571 scope.go:117] "RemoveContainer" containerID="61a59b65ff091ee819b25b8b0b09269f0f5d238be012346bae788fd9919d9be0"
Jul 17 10:02:17.610139 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:17.610109    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m_odh-ai-gateway-infra(0a16d5f4-d5aa-428c-a2f6-07b786b6babe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe"
Jul 17 10:02:18.617338 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:18.617310    2571 scope.go:117] "RemoveContainer" containerID="61a59b65ff091ee819b25b8b0b09269f0f5d238be012346bae788fd9919d9be0"
Jul 17 10:02:18.617745 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:18.617549    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m_odh-ai-gateway-infra(0a16d5f4-d5aa-428c-a2f6-07b786b6babe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe"
Jul 17 10:02:19.752349 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:19.752313    2571 scope.go:117] "RemoveContainer" containerID="64e9598a1cb9e55ffe6f70a6646c7f395d12ae381db53636350ba3b48edc0eb3"
Jul 17 10:02:20.630604 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:20.630571    2571 generic.go:356] "Generic (PLEG): container finished" podID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerID="e7b1b24a9c383fb6894ab82a4a7bbd542387600552cb0e1a4d38ec1de7fa533d" exitCode=6
Jul 17 10:02:20.630604 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:20.630600    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj" event={"ID":"d1be80a2-0748-42b4-9699-746fa28f4b3b","Type":"ContainerDied","Data":"e7b1b24a9c383fb6894ab82a4a7bbd542387600552cb0e1a4d38ec1de7fa533d"}
Jul 17 10:02:20.630848 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:20.630660    2571 scope.go:117] "RemoveContainer" containerID="64e9598a1cb9e55ffe6f70a6646c7f395d12ae381db53636350ba3b48edc0eb3"
Jul 17 10:02:20.630968 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:20.630952    2571 scope.go:117] "RemoveContainer" containerID="e7b1b24a9c383fb6894ab82a4a7bbd542387600552cb0e1a4d38ec1de7fa533d"
Jul 17 10:02:20.631239 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:20.631219    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj_odh-ai-gateway-infra(d1be80a2-0748-42b4-9699-746fa28f4b3b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b"
Jul 17 10:02:21.680381 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:21.680337    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"]
Jul 17 10:02:21.824403 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:21.824373    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"
Jul 17 10:02:21.974662 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:21.974629    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1be80a2-0748-42b4-9699-746fa28f4b3b-maas-api-service-ca\") pod \"d1be80a2-0748-42b4-9699-746fa28f4b3b\" (UID: \"d1be80a2-0748-42b4-9699-746fa28f4b3b\") "
Jul 17 10:02:21.975007 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:21.974984    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1be80a2-0748-42b4-9699-746fa28f4b3b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d1be80a2-0748-42b4-9699-746fa28f4b3b" (UID: "d1be80a2-0748-42b4-9699-746fa28f4b3b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:02:22.075746 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:22.075706    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1be80a2-0748-42b4-9699-746fa28f4b3b-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:02:22.647612 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:22.647587    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"
Jul 17 10:02:22.647780 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:22.647578    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj" event={"ID":"d1be80a2-0748-42b4-9699-746fa28f4b3b","Type":"ContainerDied","Data":"cb3b94583afbb847e5df3891e892ecabb40b2e042116115e3186e88133ac326a"}
Jul 17 10:02:22.647780 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:22.647722    2571 scope.go:117] "RemoveContainer" containerID="e7b1b24a9c383fb6894ab82a4a7bbd542387600552cb0e1a4d38ec1de7fa533d"
Jul 17 10:02:22.680717 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:22.680677    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"]
Jul 17 10:02:22.685608 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:22.685579    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-pcrqj"]
Jul 17 10:02:23.026801 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.026761    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"]
Jul 17 10:02:23.027217 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.027204    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerName="revoke-keys"
Jul 17 10:02:23.027282 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.027218    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerName="revoke-keys"
Jul 17 10:02:23.027282 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.027228    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerName="revoke-keys"
Jul 17 10:02:23.027282 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.027234    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerName="revoke-keys"
Jul 17 10:02:23.027282 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.027246    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerName="revoke-keys"
Jul 17 10:02:23.027282 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.027252    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerName="revoke-keys"
Jul 17 10:02:23.027455 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.027320    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerName="revoke-keys"
Jul 17 10:02:23.027455 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.027328    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerName="revoke-keys"
Jul 17 10:02:23.027455 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.027334    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b" containerName="revoke-keys"
Jul 17 10:02:23.031811 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.031790    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"
Jul 17 10:02:23.043629 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.043608    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"]
Jul 17 10:02:23.184877 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.184841    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f08a05db-614e-4c21-a097-15bdf151dc32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg\" (UID: \"f08a05db-614e-4c21-a097-15bdf151dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"
Jul 17 10:02:23.285960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.285877    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f08a05db-614e-4c21-a097-15bdf151dc32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg\" (UID: \"f08a05db-614e-4c21-a097-15bdf151dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"
Jul 17 10:02:23.286650 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.286615    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f08a05db-614e-4c21-a097-15bdf151dc32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg\" (UID: \"f08a05db-614e-4c21-a097-15bdf151dc32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"
Jul 17 10:02:23.341515 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.341459    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"
Jul 17 10:02:23.495117 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.495089    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"]
Jul 17 10:02:23.496979 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:02:23.496945    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf08a05db_614e_4c21_a097_15bdf151dc32.slice/crio-7578e9ec8ed9a20be45b5efc811e3d325960822b60d5bd5bb01106f2c9ccc4ce WatchSource:0}: Error finding container 7578e9ec8ed9a20be45b5efc811e3d325960822b60d5bd5bb01106f2c9ccc4ce: Status 404 returned error can't find the container with id 7578e9ec8ed9a20be45b5efc811e3d325960822b60d5bd5bb01106f2c9ccc4ce
Jul 17 10:02:23.654541 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.654495    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg" event={"ID":"f08a05db-614e-4c21-a097-15bdf151dc32","Type":"ContainerStarted","Data":"7578e9ec8ed9a20be45b5efc811e3d325960822b60d5bd5bb01106f2c9ccc4ce"}
Jul 17 10:02:23.759015 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:23.758983    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1be80a2-0748-42b4-9699-746fa28f4b3b" path="/var/lib/kubelet/pods/d1be80a2-0748-42b4-9699-746fa28f4b3b/volumes"
Jul 17 10:02:24.662443 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:24.662403    2571 generic.go:356] "Generic (PLEG): container finished" podID="f08a05db-614e-4c21-a097-15bdf151dc32" containerID="2e3ddc9065c93b710e3ab3ed0fba0aa138e04cdd5d575564c0391ce572681a16" exitCode=6
Jul 17 10:02:24.662777 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:24.662493    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg" event={"ID":"f08a05db-614e-4c21-a097-15bdf151dc32","Type":"ContainerDied","Data":"2e3ddc9065c93b710e3ab3ed0fba0aa138e04cdd5d575564c0391ce572681a16"}
Jul 17 10:02:24.662777 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:24.662702    2571 scope.go:117] "RemoveContainer" containerID="2e3ddc9065c93b710e3ab3ed0fba0aa138e04cdd5d575564c0391ce572681a16"
Jul 17 10:02:25.670321 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:25.670218    2571 generic.go:356] "Generic (PLEG): container finished" podID="f08a05db-614e-4c21-a097-15bdf151dc32" containerID="42d92c81df5403de293f66b6934bf5b8ff33c46f856296fba6defdf8efb5175c" exitCode=6
Jul 17 10:02:25.670321 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:25.670308    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg" event={"ID":"f08a05db-614e-4c21-a097-15bdf151dc32","Type":"ContainerDied","Data":"42d92c81df5403de293f66b6934bf5b8ff33c46f856296fba6defdf8efb5175c"}
Jul 17 10:02:25.670883 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:25.670396    2571 scope.go:117] "RemoveContainer" containerID="2e3ddc9065c93b710e3ab3ed0fba0aa138e04cdd5d575564c0391ce572681a16"
Jul 17 10:02:25.670883 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:25.670650    2571 scope.go:117] "RemoveContainer" containerID="42d92c81df5403de293f66b6934bf5b8ff33c46f856296fba6defdf8efb5175c"
Jul 17 10:02:25.670883 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:25.670869    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg_odh-ai-gateway-infra(f08a05db-614e-4c21-a097-15bdf151dc32)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg" podUID="f08a05db-614e-4c21-a097-15bdf151dc32"
Jul 17 10:02:26.677915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:26.677883    2571 scope.go:117] "RemoveContainer" containerID="42d92c81df5403de293f66b6934bf5b8ff33c46f856296fba6defdf8efb5175c"
Jul 17 10:02:26.678316 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:26.678093    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg_odh-ai-gateway-infra(f08a05db-614e-4c21-a097-15bdf151dc32)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg" podUID="f08a05db-614e-4c21-a097-15bdf151dc32"
Jul 17 10:02:31.752325 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:31.752289    2571 scope.go:117] "RemoveContainer" containerID="61a59b65ff091ee819b25b8b0b09269f0f5d238be012346bae788fd9919d9be0"
Jul 17 10:02:32.717037 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:32.717000    2571 generic.go:356] "Generic (PLEG): container finished" podID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerID="a5a0cd3d3bda8247c9dcdb37074304691cb80703d1f6b5b8500a777968d094a3" exitCode=6
Jul 17 10:02:32.717206 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:32.717075    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m" event={"ID":"0a16d5f4-d5aa-428c-a2f6-07b786b6babe","Type":"ContainerDied","Data":"a5a0cd3d3bda8247c9dcdb37074304691cb80703d1f6b5b8500a777968d094a3"}
Jul 17 10:02:32.717206 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:32.717132    2571 scope.go:117] "RemoveContainer" containerID="61a59b65ff091ee819b25b8b0b09269f0f5d238be012346bae788fd9919d9be0"
Jul 17 10:02:32.717421 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:32.717402    2571 scope.go:117] "RemoveContainer" containerID="a5a0cd3d3bda8247c9dcdb37074304691cb80703d1f6b5b8500a777968d094a3"
Jul 17 10:02:32.717676 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:32.717653    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m_odh-ai-gateway-infra(0a16d5f4-d5aa-428c-a2f6-07b786b6babe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe"
Jul 17 10:02:33.750929 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:33.750891    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"]
Jul 17 10:02:33.900316 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:33.900284    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"
Jul 17 10:02:33.993527 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:33.993489    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a16d5f4-d5aa-428c-a2f6-07b786b6babe-maas-api-service-ca\") pod \"0a16d5f4-d5aa-428c-a2f6-07b786b6babe\" (UID: \"0a16d5f4-d5aa-428c-a2f6-07b786b6babe\") "
Jul 17 10:02:33.993852 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:33.993827    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a16d5f4-d5aa-428c-a2f6-07b786b6babe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a16d5f4-d5aa-428c-a2f6-07b786b6babe" (UID: "0a16d5f4-d5aa-428c-a2f6-07b786b6babe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:02:34.094425 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:34.094314    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a16d5f4-d5aa-428c-a2f6-07b786b6babe-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:02:34.733087 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:34.733056    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"
Jul 17 10:02:34.733266 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:34.733099    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m" event={"ID":"0a16d5f4-d5aa-428c-a2f6-07b786b6babe","Type":"ContainerDied","Data":"97970d8ba42060548e191adf14caea041376ed308b4e245a192ecc1b6fbdf29e"}
Jul 17 10:02:34.733266 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:34.733189    2571 scope.go:117] "RemoveContainer" containerID="a5a0cd3d3bda8247c9dcdb37074304691cb80703d1f6b5b8500a777968d094a3"
Jul 17 10:02:34.756372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:34.756334    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"]
Jul 17 10:02:34.762787 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:34.762754    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-snz7m"]
Jul 17 10:02:35.485314 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.485269    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"]
Jul 17 10:02:35.485756 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.485740    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerName="revoke-keys"
Jul 17 10:02:35.485756 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.485757    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerName="revoke-keys"
Jul 17 10:02:35.485876 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.485782    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerName="revoke-keys"
Jul 17 10:02:35.485876 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.485787    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerName="revoke-keys"
Jul 17 10:02:35.485876 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.485853    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerName="revoke-keys"
Jul 17 10:02:35.485876 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.485864    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerName="revoke-keys"
Jul 17 10:02:35.490738 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.490722    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"
Jul 17 10:02:35.499060 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.498479    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"]
Jul 17 10:02:35.608734 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.608698    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37707676-5761-463c-bb05-1647869712d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675\" (UID: \"37707676-5761-463c-bb05-1647869712d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"
Jul 17 10:02:35.709790 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.709751    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37707676-5761-463c-bb05-1647869712d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675\" (UID: \"37707676-5761-463c-bb05-1647869712d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"
Jul 17 10:02:35.710429 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.710410    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37707676-5761-463c-bb05-1647869712d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675\" (UID: \"37707676-5761-463c-bb05-1647869712d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"
Jul 17 10:02:35.758764 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.758683    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" path="/var/lib/kubelet/pods/0a16d5f4-d5aa-428c-a2f6-07b786b6babe/volumes"
Jul 17 10:02:35.803260 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.803225    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"
Jul 17 10:02:35.944425 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:35.944392    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"]
Jul 17 10:02:35.945817 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:02:35.945786    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37707676_5761_463c_bb05_1647869712d8.slice/crio-cfaf79686a545c3538f4eb988b2309e5e41faec0a697eb25e7d6abcd583b2466 WatchSource:0}: Error finding container cfaf79686a545c3538f4eb988b2309e5e41faec0a697eb25e7d6abcd583b2466: Status 404 returned error can't find the container with id cfaf79686a545c3538f4eb988b2309e5e41faec0a697eb25e7d6abcd583b2466
Jul 17 10:02:36.747994 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:36.747966    2571 generic.go:356] "Generic (PLEG): container finished" podID="37707676-5761-463c-bb05-1647869712d8" containerID="c6f8164978bc5e51f9cc44632b77ce2babccb9370a2172b06d5f2eb55fc3041d" exitCode=6
Jul 17 10:02:36.748129 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:36.748031    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675" event={"ID":"37707676-5761-463c-bb05-1647869712d8","Type":"ContainerDied","Data":"c6f8164978bc5e51f9cc44632b77ce2babccb9370a2172b06d5f2eb55fc3041d"}
Jul 17 10:02:36.748129 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:36.748087    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675" event={"ID":"37707676-5761-463c-bb05-1647869712d8","Type":"ContainerStarted","Data":"cfaf79686a545c3538f4eb988b2309e5e41faec0a697eb25e7d6abcd583b2466"}
Jul 17 10:02:36.748219 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:36.748205    2571 scope.go:117] "RemoveContainer" containerID="c6f8164978bc5e51f9cc44632b77ce2babccb9370a2172b06d5f2eb55fc3041d"
Jul 17 10:02:37.757843 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:37.757810    2571 scope.go:117] "RemoveContainer" containerID="42d92c81df5403de293f66b6934bf5b8ff33c46f856296fba6defdf8efb5175c"
Jul 17 10:02:37.762324 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:37.762299    2571 generic.go:356] "Generic (PLEG): container finished" podID="37707676-5761-463c-bb05-1647869712d8" containerID="fe39809235bab6509c3727d2bef2df7394ea046a159a0d1079808a2999a9bf7c" exitCode=6
Jul 17 10:02:37.762732 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:37.762713    2571 scope.go:117] "RemoveContainer" containerID="fe39809235bab6509c3727d2bef2df7394ea046a159a0d1079808a2999a9bf7c"
Jul 17 10:02:37.762950 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:37.762927    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675_odh-ai-gateway-infra(37707676-5761-463c-bb05-1647869712d8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675" podUID="37707676-5761-463c-bb05-1647869712d8"
Jul 17 10:02:37.763122 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:37.763104    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675" event={"ID":"37707676-5761-463c-bb05-1647869712d8","Type":"ContainerDied","Data":"fe39809235bab6509c3727d2bef2df7394ea046a159a0d1079808a2999a9bf7c"}
Jul 17 10:02:37.763161 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:37.763154    2571 scope.go:117] "RemoveContainer" containerID="c6f8164978bc5e51f9cc44632b77ce2babccb9370a2172b06d5f2eb55fc3041d"
Jul 17 10:02:38.770194 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:38.770159    2571 generic.go:356] "Generic (PLEG): container finished" podID="f08a05db-614e-4c21-a097-15bdf151dc32" containerID="cd9a9ccb2fee3da3eec923e490d3f6958f576138671ebef71aabcf39dbced4c1" exitCode=6
Jul 17 10:02:38.770704 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:38.770232    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg" event={"ID":"f08a05db-614e-4c21-a097-15bdf151dc32","Type":"ContainerDied","Data":"cd9a9ccb2fee3da3eec923e490d3f6958f576138671ebef71aabcf39dbced4c1"}
Jul 17 10:02:38.770704 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:38.770303    2571 scope.go:117] "RemoveContainer" containerID="42d92c81df5403de293f66b6934bf5b8ff33c46f856296fba6defdf8efb5175c"
Jul 17 10:02:38.770704 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:38.770620    2571 scope.go:117] "RemoveContainer" containerID="cd9a9ccb2fee3da3eec923e490d3f6958f576138671ebef71aabcf39dbced4c1"
Jul 17 10:02:38.770914 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:38.770892    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg_odh-ai-gateway-infra(f08a05db-614e-4c21-a097-15bdf151dc32)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg" podUID="f08a05db-614e-4c21-a097-15bdf151dc32"
Jul 17 10:02:38.772398 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:38.772380    2571 scope.go:117] "RemoveContainer" containerID="fe39809235bab6509c3727d2bef2df7394ea046a159a0d1079808a2999a9bf7c"
Jul 17 10:02:38.772581 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:38.772564    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675_odh-ai-gateway-infra(37707676-5761-463c-bb05-1647869712d8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675" podUID="37707676-5761-463c-bb05-1647869712d8"
Jul 17 10:02:39.816943 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:39.816908    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"]
Jul 17 10:02:39.959578 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:39.959555    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"
Jul 17 10:02:40.149513 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:40.149397    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f08a05db-614e-4c21-a097-15bdf151dc32-maas-api-service-ca\") pod \"f08a05db-614e-4c21-a097-15bdf151dc32\" (UID: \"f08a05db-614e-4c21-a097-15bdf151dc32\") "
Jul 17 10:02:40.149787 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:40.149765    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f08a05db-614e-4c21-a097-15bdf151dc32-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f08a05db-614e-4c21-a097-15bdf151dc32" (UID: "f08a05db-614e-4c21-a097-15bdf151dc32"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:02:40.250431 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:40.250396    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f08a05db-614e-4c21-a097-15bdf151dc32-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:02:40.786752 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:40.786722    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"
Jul 17 10:02:40.786929 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:40.786766    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg" event={"ID":"f08a05db-614e-4c21-a097-15bdf151dc32","Type":"ContainerDied","Data":"7578e9ec8ed9a20be45b5efc811e3d325960822b60d5bd5bb01106f2c9ccc4ce"}
Jul 17 10:02:40.786929 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:40.786828    2571 scope.go:117] "RemoveContainer" containerID="cd9a9ccb2fee3da3eec923e490d3f6958f576138671ebef71aabcf39dbced4c1"
Jul 17 10:02:40.816884 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:40.816852    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"]
Jul 17 10:02:40.819410 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:40.819385    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-2mswg"]
Jul 17 10:02:41.758383 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:41.758345    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f08a05db-614e-4c21-a097-15bdf151dc32" path="/var/lib/kubelet/pods/f08a05db-614e-4c21-a097-15bdf151dc32/volumes"
Jul 17 10:02:49.752526 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:49.752487    2571 scope.go:117] "RemoveContainer" containerID="fe39809235bab6509c3727d2bef2df7394ea046a159a0d1079808a2999a9bf7c"
Jul 17 10:02:50.851071 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:50.851036    2571 generic.go:356] "Generic (PLEG): container finished" podID="37707676-5761-463c-bb05-1647869712d8" containerID="671bec878559a0a6b68e86e02414bb5f47072cca53fbc74d0934d75584425320" exitCode=6
Jul 17 10:02:50.851504 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:50.851078    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675" event={"ID":"37707676-5761-463c-bb05-1647869712d8","Type":"ContainerDied","Data":"671bec878559a0a6b68e86e02414bb5f47072cca53fbc74d0934d75584425320"}
Jul 17 10:02:50.851504 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:50.851125    2571 scope.go:117] "RemoveContainer" containerID="fe39809235bab6509c3727d2bef2df7394ea046a159a0d1079808a2999a9bf7c"
Jul 17 10:02:50.851504 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:50.851425    2571 scope.go:117] "RemoveContainer" containerID="671bec878559a0a6b68e86e02414bb5f47072cca53fbc74d0934d75584425320"
Jul 17 10:02:50.851709 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:02:50.851687    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675_odh-ai-gateway-infra(37707676-5761-463c-bb05-1647869712d8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675" podUID="37707676-5761-463c-bb05-1647869712d8"
Jul 17 10:02:51.884241 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:51.884204    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"]
Jul 17 10:02:52.029264 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:52.029235    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"
Jul 17 10:02:52.055975 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:52.055937    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37707676-5761-463c-bb05-1647869712d8-maas-api-service-ca\") pod \"37707676-5761-463c-bb05-1647869712d8\" (UID: \"37707676-5761-463c-bb05-1647869712d8\") "
Jul 17 10:02:52.056308 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:52.056284    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37707676-5761-463c-bb05-1647869712d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37707676-5761-463c-bb05-1647869712d8" (UID: "37707676-5761-463c-bb05-1647869712d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:02:52.157506 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:52.157386    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37707676-5761-463c-bb05-1647869712d8-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:02:52.868880 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:52.868844    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"
Jul 17 10:02:52.869086 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:52.868847    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675" event={"ID":"37707676-5761-463c-bb05-1647869712d8","Type":"ContainerDied","Data":"cfaf79686a545c3538f4eb988b2309e5e41faec0a697eb25e7d6abcd583b2466"}
Jul 17 10:02:52.869086 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:52.869009    2571 scope.go:117] "RemoveContainer" containerID="671bec878559a0a6b68e86e02414bb5f47072cca53fbc74d0934d75584425320"
Jul 17 10:02:52.905242 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:52.905205    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"]
Jul 17 10:02:52.909701 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:52.909663    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-4c675"]
Jul 17 10:02:53.758981 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:02:53.758949    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37707676-5761-463c-bb05-1647869712d8" path="/var/lib/kubelet/pods/37707676-5761-463c-bb05-1647869712d8/volumes"
Jul 17 10:03:05.229895 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.229855    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"]
Jul 17 10:03:05.230393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230341    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerName="revoke-keys"
Jul 17 10:03:05.230393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230353    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerName="revoke-keys"
Jul 17 10:03:05.230393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230363    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37707676-5761-463c-bb05-1647869712d8" containerName="revoke-keys"
Jul 17 10:03:05.230393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230369    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="37707676-5761-463c-bb05-1647869712d8" containerName="revoke-keys"
Jul 17 10:03:05.230393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230380    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f08a05db-614e-4c21-a097-15bdf151dc32" containerName="revoke-keys"
Jul 17 10:03:05.230393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230390    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f08a05db-614e-4c21-a097-15bdf151dc32" containerName="revoke-keys"
Jul 17 10:03:05.230641 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230404    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f08a05db-614e-4c21-a097-15bdf151dc32" containerName="revoke-keys"
Jul 17 10:03:05.230641 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230409    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f08a05db-614e-4c21-a097-15bdf151dc32" containerName="revoke-keys"
Jul 17 10:03:05.230641 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230429    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37707676-5761-463c-bb05-1647869712d8" containerName="revoke-keys"
Jul 17 10:03:05.230641 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230434    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="37707676-5761-463c-bb05-1647869712d8" containerName="revoke-keys"
Jul 17 10:03:05.230641 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230515    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f08a05db-614e-4c21-a097-15bdf151dc32" containerName="revoke-keys"
Jul 17 10:03:05.230641 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230523    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="37707676-5761-463c-bb05-1647869712d8" containerName="revoke-keys"
Jul 17 10:03:05.230641 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230531    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f08a05db-614e-4c21-a097-15bdf151dc32" containerName="revoke-keys"
Jul 17 10:03:05.230641 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230536    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a16d5f4-d5aa-428c-a2f6-07b786b6babe" containerName="revoke-keys"
Jul 17 10:03:05.230641 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.230548    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="37707676-5761-463c-bb05-1647869712d8" containerName="revoke-keys"
Jul 17 10:03:05.235320 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.235300    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"
Jul 17 10:03:05.238671 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.238652    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:03:05.244776 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.244748    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"]
Jul 17 10:03:05.382778 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.382734    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d09f287f-4e03-4fd0-8718-59c50f17b9df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds\" (UID: \"d09f287f-4e03-4fd0-8718-59c50f17b9df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"
Jul 17 10:03:05.484021 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.483913    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d09f287f-4e03-4fd0-8718-59c50f17b9df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds\" (UID: \"d09f287f-4e03-4fd0-8718-59c50f17b9df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"
Jul 17 10:03:05.484599 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.484577    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d09f287f-4e03-4fd0-8718-59c50f17b9df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds\" (UID: \"d09f287f-4e03-4fd0-8718-59c50f17b9df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"
Jul 17 10:03:05.547122 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.547082    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"
Jul 17 10:03:05.690672 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.690639    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"]
Jul 17 10:03:05.692226 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:03:05.692196    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09f287f_4e03_4fd0_8718_59c50f17b9df.slice/crio-c69f791515aedc86b5a3633a6d823570f313930362e024c0170219cf20f7a78a WatchSource:0}: Error finding container c69f791515aedc86b5a3633a6d823570f313930362e024c0170219cf20f7a78a: Status 404 returned error can't find the container with id c69f791515aedc86b5a3633a6d823570f313930362e024c0170219cf20f7a78a
Jul 17 10:03:05.944963 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:05.944924    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds" event={"ID":"d09f287f-4e03-4fd0-8718-59c50f17b9df","Type":"ContainerStarted","Data":"c69f791515aedc86b5a3633a6d823570f313930362e024c0170219cf20f7a78a"}
Jul 17 10:03:06.952270 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:06.952227    2571 generic.go:356] "Generic (PLEG): container finished" podID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerID="c15cc2cab5151aa629658a04ef304c3e68978e20e7775130f6090051db6a966b" exitCode=6
Jul 17 10:03:06.952700 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:06.952316    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds" event={"ID":"d09f287f-4e03-4fd0-8718-59c50f17b9df","Type":"ContainerDied","Data":"c15cc2cab5151aa629658a04ef304c3e68978e20e7775130f6090051db6a966b"}
Jul 17 10:03:06.952700 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:06.952518    2571 scope.go:117] "RemoveContainer" containerID="c15cc2cab5151aa629658a04ef304c3e68978e20e7775130f6090051db6a966b"
Jul 17 10:03:07.959543 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:07.959502    2571 generic.go:356] "Generic (PLEG): container finished" podID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerID="33b306c75abb914566cde54ba9b8201117c6a1f09e7ae3f6d0eba7944cee700b" exitCode=6
Jul 17 10:03:07.960024 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:07.959596    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds" event={"ID":"d09f287f-4e03-4fd0-8718-59c50f17b9df","Type":"ContainerDied","Data":"33b306c75abb914566cde54ba9b8201117c6a1f09e7ae3f6d0eba7944cee700b"}
Jul 17 10:03:07.960024 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:07.959673    2571 scope.go:117] "RemoveContainer" containerID="c15cc2cab5151aa629658a04ef304c3e68978e20e7775130f6090051db6a966b"
Jul 17 10:03:07.960024 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:07.959879    2571 scope.go:117] "RemoveContainer" containerID="33b306c75abb914566cde54ba9b8201117c6a1f09e7ae3f6d0eba7944cee700b"
Jul 17 10:03:07.960153 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:03:07.960128    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds_odh-ai-gateway-infra(d09f287f-4e03-4fd0-8718-59c50f17b9df)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df"
Jul 17 10:03:08.967564 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:08.967534    2571 scope.go:117] "RemoveContainer" containerID="33b306c75abb914566cde54ba9b8201117c6a1f09e7ae3f6d0eba7944cee700b"
Jul 17 10:03:08.967962 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:03:08.967743    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds_odh-ai-gateway-infra(d09f287f-4e03-4fd0-8718-59c50f17b9df)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df"
Jul 17 10:03:13.068437 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.068393    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"]
Jul 17 10:03:13.069072 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.069051    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37707676-5761-463c-bb05-1647869712d8" containerName="revoke-keys"
Jul 17 10:03:13.069179 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.069074    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="37707676-5761-463c-bb05-1647869712d8" containerName="revoke-keys"
Jul 17 10:03:13.069179 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.069096    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f08a05db-614e-4c21-a097-15bdf151dc32" containerName="revoke-keys"
Jul 17 10:03:13.069179 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.069104    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f08a05db-614e-4c21-a097-15bdf151dc32" containerName="revoke-keys"
Jul 17 10:03:13.069341 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.069223    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f08a05db-614e-4c21-a097-15bdf151dc32" containerName="revoke-keys"
Jul 17 10:03:13.069341 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.069237    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="37707676-5761-463c-bb05-1647869712d8" containerName="revoke-keys"
Jul 17 10:03:13.073564 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.073543    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"
Jul 17 10:03:13.080832 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.080801    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"]
Jul 17 10:03:13.156302 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.156260    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b09e6c03-676d-4e2e-b776-c08a596bb2a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn\" (UID: \"b09e6c03-676d-4e2e-b776-c08a596bb2a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"
Jul 17 10:03:13.256894 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.256847    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b09e6c03-676d-4e2e-b776-c08a596bb2a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn\" (UID: \"b09e6c03-676d-4e2e-b776-c08a596bb2a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"
Jul 17 10:03:13.257511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.257491    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b09e6c03-676d-4e2e-b776-c08a596bb2a9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn\" (UID: \"b09e6c03-676d-4e2e-b776-c08a596bb2a9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"
Jul 17 10:03:13.385549 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.385435    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"
Jul 17 10:03:13.533697 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.533670    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"]
Jul 17 10:03:13.535847 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:03:13.535818    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb09e6c03_676d_4e2e_b776_c08a596bb2a9.slice/crio-64e6a0028f90c59b86a204393a4cb324fa616f2d7ab0c6ec39929cd2e28663d8 WatchSource:0}: Error finding container 64e6a0028f90c59b86a204393a4cb324fa616f2d7ab0c6ec39929cd2e28663d8: Status 404 returned error can't find the container with id 64e6a0028f90c59b86a204393a4cb324fa616f2d7ab0c6ec39929cd2e28663d8
Jul 17 10:03:13.997567 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.997535    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn" event={"ID":"b09e6c03-676d-4e2e-b776-c08a596bb2a9","Type":"ContainerStarted","Data":"f16cea8724690be47884b974f7a13f02fc54e9328959678ae2a1b61995e09eeb"}
Jul 17 10:03:13.997754 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:13.997590    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn" event={"ID":"b09e6c03-676d-4e2e-b776-c08a596bb2a9","Type":"ContainerStarted","Data":"64e6a0028f90c59b86a204393a4cb324fa616f2d7ab0c6ec39929cd2e28663d8"}
Jul 17 10:03:14.016237 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:14.016120    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn" podStartSLOduration=1.016103605 podStartE2EDuration="1.016103605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:03:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:03:14.013284535 +0000 UTC m=+2678.078105259" watchObservedRunningTime="2026-07-17 10:03:14.016103605 +0000 UTC m=+2678.080924326"
Jul 17 10:03:15.004847 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:15.004812    2571 generic.go:356] "Generic (PLEG): container finished" podID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerID="f16cea8724690be47884b974f7a13f02fc54e9328959678ae2a1b61995e09eeb" exitCode=6
Jul 17 10:03:15.005235 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:15.004878    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn" event={"ID":"b09e6c03-676d-4e2e-b776-c08a596bb2a9","Type":"ContainerDied","Data":"f16cea8724690be47884b974f7a13f02fc54e9328959678ae2a1b61995e09eeb"}
Jul 17 10:03:15.005235 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:15.005122    2571 scope.go:117] "RemoveContainer" containerID="f16cea8724690be47884b974f7a13f02fc54e9328959678ae2a1b61995e09eeb"
Jul 17 10:03:16.012715 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:16.012678    2571 generic.go:356] "Generic (PLEG): container finished" podID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerID="5c6ad6e23c89a5ad1be3fb8b0075da322138014509f1d5996c2926aac1be9eda" exitCode=6
Jul 17 10:03:16.013151 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:16.012762    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn" event={"ID":"b09e6c03-676d-4e2e-b776-c08a596bb2a9","Type":"ContainerDied","Data":"5c6ad6e23c89a5ad1be3fb8b0075da322138014509f1d5996c2926aac1be9eda"}
Jul 17 10:03:16.013151 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:16.012826    2571 scope.go:117] "RemoveContainer" containerID="f16cea8724690be47884b974f7a13f02fc54e9328959678ae2a1b61995e09eeb"
Jul 17 10:03:16.013151 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:16.013112    2571 scope.go:117] "RemoveContainer" containerID="5c6ad6e23c89a5ad1be3fb8b0075da322138014509f1d5996c2926aac1be9eda"
Jul 17 10:03:16.013363 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:03:16.013331    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn_odh-ai-gateway-infra(b09e6c03-676d-4e2e-b776-c08a596bb2a9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9"
Jul 17 10:03:17.020683 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:17.020656    2571 scope.go:117] "RemoveContainer" containerID="5c6ad6e23c89a5ad1be3fb8b0075da322138014509f1d5996c2926aac1be9eda"
Jul 17 10:03:17.021059 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:03:17.020847    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn_odh-ai-gateway-infra(b09e6c03-676d-4e2e-b776-c08a596bb2a9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9"
Jul 17 10:03:23.752194 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:23.752161    2571 scope.go:117] "RemoveContainer" containerID="33b306c75abb914566cde54ba9b8201117c6a1f09e7ae3f6d0eba7944cee700b"
Jul 17 10:03:24.062332 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:24.062299    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds" event={"ID":"d09f287f-4e03-4fd0-8718-59c50f17b9df","Type":"ContainerStarted","Data":"0cd548f9efa352839a4dd9098443852b23645e80d4d83c3e57628ee55ffb46e9"}
Jul 17 10:03:24.081763 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:24.081712    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds" podStartSLOduration=19.081697175 podStartE2EDuration="19.081697175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:03:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:03:24.080836315 +0000 UTC m=+2688.145657038" watchObservedRunningTime="2026-07-17 10:03:24.081697175 +0000 UTC m=+2688.146517897"
Jul 17 10:03:25.069638 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.069608    2571 generic.go:356] "Generic (PLEG): container finished" podID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerID="0cd548f9efa352839a4dd9098443852b23645e80d4d83c3e57628ee55ffb46e9" exitCode=6
Jul 17 10:03:25.070093 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.069653    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds" event={"ID":"d09f287f-4e03-4fd0-8718-59c50f17b9df","Type":"ContainerDied","Data":"0cd548f9efa352839a4dd9098443852b23645e80d4d83c3e57628ee55ffb46e9"}
Jul 17 10:03:25.070093 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.069701    2571 scope.go:117] "RemoveContainer" containerID="33b306c75abb914566cde54ba9b8201117c6a1f09e7ae3f6d0eba7944cee700b"
Jul 17 10:03:25.070093 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.069945    2571 scope.go:117] "RemoveContainer" containerID="0cd548f9efa352839a4dd9098443852b23645e80d4d83c3e57628ee55ffb46e9"
Jul 17 10:03:25.070233 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:03:25.070196    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds_odh-ai-gateway-infra(d09f287f-4e03-4fd0-8718-59c50f17b9df)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df"
Jul 17 10:03:25.093793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.093756    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"]
Jul 17 10:03:25.532513 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.532457    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"]
Jul 17 10:03:25.537533 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.537514    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"
Jul 17 10:03:25.544800 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.544766    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"]
Jul 17 10:03:25.669048 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.669003    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d9f8c26-e8a7-4843-9ee4-a9b947e71e27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7\" (UID: \"2d9f8c26-e8a7-4843-9ee4-a9b947e71e27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"
Jul 17 10:03:25.769612 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.769577    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d9f8c26-e8a7-4843-9ee4-a9b947e71e27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7\" (UID: \"2d9f8c26-e8a7-4843-9ee4-a9b947e71e27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"
Jul 17 10:03:25.770224 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.770202    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d9f8c26-e8a7-4843-9ee4-a9b947e71e27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7\" (UID: \"2d9f8c26-e8a7-4843-9ee4-a9b947e71e27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"
Jul 17 10:03:25.851152 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.851065    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"
Jul 17 10:03:25.995152 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:25.995124    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"]
Jul 17 10:03:25.996243 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:03:25.996215    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9f8c26_e8a7_4843_9ee4_a9b947e71e27.slice/crio-a4dff711cf058cdda726212e3e2909ca154803594d2eb39f71c35dcbfb15f9f5 WatchSource:0}: Error finding container a4dff711cf058cdda726212e3e2909ca154803594d2eb39f71c35dcbfb15f9f5: Status 404 returned error can't find the container with id a4dff711cf058cdda726212e3e2909ca154803594d2eb39f71c35dcbfb15f9f5
Jul 17 10:03:26.076481 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:26.076428    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7" event={"ID":"2d9f8c26-e8a7-4843-9ee4-a9b947e71e27","Type":"ContainerStarted","Data":"a4dff711cf058cdda726212e3e2909ca154803594d2eb39f71c35dcbfb15f9f5"}
Jul 17 10:03:26.212831 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:26.212802    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"
Jul 17 10:03:26.375481 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:26.375436    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d09f287f-4e03-4fd0-8718-59c50f17b9df-maas-api-service-ca\") pod \"d09f287f-4e03-4fd0-8718-59c50f17b9df\" (UID: \"d09f287f-4e03-4fd0-8718-59c50f17b9df\") "
Jul 17 10:03:26.375783 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:26.375761    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d09f287f-4e03-4fd0-8718-59c50f17b9df-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d09f287f-4e03-4fd0-8718-59c50f17b9df" (UID: "d09f287f-4e03-4fd0-8718-59c50f17b9df"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:03:26.476755 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:26.476721    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d09f287f-4e03-4fd0-8718-59c50f17b9df-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:03:27.086133 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:27.086093    2571 generic.go:356] "Generic (PLEG): container finished" podID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerID="750d9c91c6ef097cf328730e3839e90f4957cffd8f58e5b527b6a24521cbc8ec" exitCode=6
Jul 17 10:03:27.086651 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:27.086175    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7" event={"ID":"2d9f8c26-e8a7-4843-9ee4-a9b947e71e27","Type":"ContainerDied","Data":"750d9c91c6ef097cf328730e3839e90f4957cffd8f58e5b527b6a24521cbc8ec"}
Jul 17 10:03:27.086651 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:27.086397    2571 scope.go:117] "RemoveContainer" containerID="750d9c91c6ef097cf328730e3839e90f4957cffd8f58e5b527b6a24521cbc8ec"
Jul 17 10:03:27.087799 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:27.087780    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"
Jul 17 10:03:27.087925 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:27.087822    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds" event={"ID":"d09f287f-4e03-4fd0-8718-59c50f17b9df","Type":"ContainerDied","Data":"c69f791515aedc86b5a3633a6d823570f313930362e024c0170219cf20f7a78a"}
Jul 17 10:03:27.087925 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:27.087885    2571 scope.go:117] "RemoveContainer" containerID="0cd548f9efa352839a4dd9098443852b23645e80d4d83c3e57628ee55ffb46e9"
Jul 17 10:03:27.117590 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:27.117563    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"]
Jul 17 10:03:27.124646 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:27.124623    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-jwjds"]
Jul 17 10:03:27.755705 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:27.755673    2571 scope.go:117] "RemoveContainer" containerID="5c6ad6e23c89a5ad1be3fb8b0075da322138014509f1d5996c2926aac1be9eda"
Jul 17 10:03:27.759961 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:27.759940    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df" path="/var/lib/kubelet/pods/d09f287f-4e03-4fd0-8718-59c50f17b9df/volumes"
Jul 17 10:03:28.096842 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:28.096804    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn" event={"ID":"b09e6c03-676d-4e2e-b776-c08a596bb2a9","Type":"ContainerStarted","Data":"eff7d5ea730089a98161fcbfb69a2038404224349fcde397afe88576efa2e852"}
Jul 17 10:03:28.098767 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:28.098729    2571 generic.go:356] "Generic (PLEG): container finished" podID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerID="1d0fa837e6d8cb162e6d4e314144082dbcddf4133f350822ae95a3108d485d82" exitCode=6
Jul 17 10:03:28.098902 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:28.098764    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7" event={"ID":"2d9f8c26-e8a7-4843-9ee4-a9b947e71e27","Type":"ContainerDied","Data":"1d0fa837e6d8cb162e6d4e314144082dbcddf4133f350822ae95a3108d485d82"}
Jul 17 10:03:28.098902 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:28.098815    2571 scope.go:117] "RemoveContainer" containerID="750d9c91c6ef097cf328730e3839e90f4957cffd8f58e5b527b6a24521cbc8ec"
Jul 17 10:03:28.099005 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:28.098975    2571 scope.go:117] "RemoveContainer" containerID="1d0fa837e6d8cb162e6d4e314144082dbcddf4133f350822ae95a3108d485d82"
Jul 17 10:03:28.099170 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:03:28.099152    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7_odh-ai-gateway-infra(2d9f8c26-e8a7-4843-9ee4-a9b947e71e27)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27"
Jul 17 10:03:29.106272 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:29.106236    2571 generic.go:356] "Generic (PLEG): container finished" podID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerID="eff7d5ea730089a98161fcbfb69a2038404224349fcde397afe88576efa2e852" exitCode=6
Jul 17 10:03:29.106770 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:29.106311    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn" event={"ID":"b09e6c03-676d-4e2e-b776-c08a596bb2a9","Type":"ContainerDied","Data":"eff7d5ea730089a98161fcbfb69a2038404224349fcde397afe88576efa2e852"}
Jul 17 10:03:29.106770 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:29.106413    2571 scope.go:117] "RemoveContainer" containerID="5c6ad6e23c89a5ad1be3fb8b0075da322138014509f1d5996c2926aac1be9eda"
Jul 17 10:03:29.106770 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:29.106757    2571 scope.go:117] "RemoveContainer" containerID="eff7d5ea730089a98161fcbfb69a2038404224349fcde397afe88576efa2e852"
Jul 17 10:03:29.107031 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:03:29.107008    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn_odh-ai-gateway-infra(b09e6c03-676d-4e2e-b776-c08a596bb2a9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9"
Jul 17 10:03:29.108215 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:29.108194    2571 scope.go:117] "RemoveContainer" containerID="1d0fa837e6d8cb162e6d4e314144082dbcddf4133f350822ae95a3108d485d82"
Jul 17 10:03:29.108360 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:03:29.108342    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7_odh-ai-gateway-infra(2d9f8c26-e8a7-4843-9ee4-a9b947e71e27)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27"
Jul 17 10:03:29.128446 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:29.128424    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"]
Jul 17 10:03:30.258513 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:30.258482    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"
Jul 17 10:03:30.416560 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:30.416430    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b09e6c03-676d-4e2e-b776-c08a596bb2a9-maas-api-service-ca\") pod \"b09e6c03-676d-4e2e-b776-c08a596bb2a9\" (UID: \"b09e6c03-676d-4e2e-b776-c08a596bb2a9\") "
Jul 17 10:03:30.416858 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:30.416833    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b09e6c03-676d-4e2e-b776-c08a596bb2a9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b09e6c03-676d-4e2e-b776-c08a596bb2a9" (UID: "b09e6c03-676d-4e2e-b776-c08a596bb2a9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:03:30.518063 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:30.518022    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b09e6c03-676d-4e2e-b776-c08a596bb2a9-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:03:31.122274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:31.122240    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn" event={"ID":"b09e6c03-676d-4e2e-b776-c08a596bb2a9","Type":"ContainerDied","Data":"64e6a0028f90c59b86a204393a4cb324fa616f2d7ab0c6ec39929cd2e28663d8"}
Jul 17 10:03:31.122508 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:31.122297    2571 scope.go:117] "RemoveContainer" containerID="eff7d5ea730089a98161fcbfb69a2038404224349fcde397afe88576efa2e852"
Jul 17 10:03:31.122508 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:31.122252    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"
Jul 17 10:03:31.145849 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:31.145819    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"]
Jul 17 10:03:31.151078 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:31.151048    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-9rdmn"]
Jul 17 10:03:31.759075 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:31.759040    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" path="/var/lib/kubelet/pods/b09e6c03-676d-4e2e-b776-c08a596bb2a9/volumes"
Jul 17 10:03:37.590445 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:37.590412    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 10:03:37.600232 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:37.600207    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 10:03:39.752498 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:39.752439    2571 scope.go:117] "RemoveContainer" containerID="1d0fa837e6d8cb162e6d4e314144082dbcddf4133f350822ae95a3108d485d82"
Jul 17 10:03:40.178742 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:40.178658    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7" event={"ID":"2d9f8c26-e8a7-4843-9ee4-a9b947e71e27","Type":"ContainerStarted","Data":"da2ff550a10dbfe588caf460485652bc89c4f22e5519d6a302c1024ebb9eed91"}
Jul 17 10:03:40.195796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:40.195729    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7" podStartSLOduration=15.195711904 podStartE2EDuration="15.195711904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:03:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:03:40.19439334 +0000 UTC m=+2704.259214063" watchObservedRunningTime="2026-07-17 10:03:40.195711904 +0000 UTC m=+2704.260532627"
Jul 17 10:03:41.186877 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:41.186845    2571 generic.go:356] "Generic (PLEG): container finished" podID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerID="da2ff550a10dbfe588caf460485652bc89c4f22e5519d6a302c1024ebb9eed91" exitCode=6
Jul 17 10:03:41.187297 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:41.186917    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7" event={"ID":"2d9f8c26-e8a7-4843-9ee4-a9b947e71e27","Type":"ContainerDied","Data":"da2ff550a10dbfe588caf460485652bc89c4f22e5519d6a302c1024ebb9eed91"}
Jul 17 10:03:41.187297 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:41.186978    2571 scope.go:117] "RemoveContainer" containerID="1d0fa837e6d8cb162e6d4e314144082dbcddf4133f350822ae95a3108d485d82"
Jul 17 10:03:41.187297 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:41.187270    2571 scope.go:117] "RemoveContainer" containerID="da2ff550a10dbfe588caf460485652bc89c4f22e5519d6a302c1024ebb9eed91"
Jul 17 10:03:41.187507 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:03:41.187488    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7_odh-ai-gateway-infra(2d9f8c26-e8a7-4843-9ee4-a9b947e71e27)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27"
Jul 17 10:03:41.226263 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:41.226231    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"]
Jul 17 10:03:42.341537 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:42.341512    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"
Jul 17 10:03:42.425456 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:42.425414    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d9f8c26-e8a7-4843-9ee4-a9b947e71e27-maas-api-service-ca\") pod \"2d9f8c26-e8a7-4843-9ee4-a9b947e71e27\" (UID: \"2d9f8c26-e8a7-4843-9ee4-a9b947e71e27\") "
Jul 17 10:03:42.425802 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:42.425776    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d9f8c26-e8a7-4843-9ee4-a9b947e71e27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" (UID: "2d9f8c26-e8a7-4843-9ee4-a9b947e71e27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:03:42.527023 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:42.526935    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d9f8c26-e8a7-4843-9ee4-a9b947e71e27-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:03:43.201139 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.201102    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"
Jul 17 10:03:43.201139 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.201134    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7" event={"ID":"2d9f8c26-e8a7-4843-9ee4-a9b947e71e27","Type":"ContainerDied","Data":"a4dff711cf058cdda726212e3e2909ca154803594d2eb39f71c35dcbfb15f9f5"}
Jul 17 10:03:43.201384 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.201192    2571 scope.go:117] "RemoveContainer" containerID="da2ff550a10dbfe588caf460485652bc89c4f22e5519d6a302c1024ebb9eed91"
Jul 17 10:03:43.225953 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.225924    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"]
Jul 17 10:03:43.230654 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.230630    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vt5l7"]
Jul 17 10:03:43.265297 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265252    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs"]
Jul 17 10:03:43.265770 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265752    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerName="revoke-keys"
Jul 17 10:03:43.265864 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265772    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerName="revoke-keys"
Jul 17 10:03:43.265864 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265790    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerName="revoke-keys"
Jul 17 10:03:43.265864 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265799    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerName="revoke-keys"
Jul 17 10:03:43.265864 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265809    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerName="revoke-keys"
Jul 17 10:03:43.265864 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265818    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerName="revoke-keys"
Jul 17 10:03:43.265864 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265848    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerName="revoke-keys"
Jul 17 10:03:43.265864 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265858    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265868    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265876    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265889    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265898    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265910    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265918    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265933    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.265942    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.266045    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.266057    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.266068    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.266078    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.266087    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.266095    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.266103    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" containerName="revoke-keys"
Jul 17 10:03:43.266204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.266114    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerName="revoke-keys"
Jul 17 10:03:43.270758 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.270736    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs"
Jul 17 10:03:43.273360 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.273341    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:03:43.278429 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.278404    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs"]
Jul 17 10:03:43.333289 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.333248    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb555f53-125f-41ac-8e46-c4ce4bf40eb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-43c677b2-fznxs\" (UID: \"bb555f53-125f-41ac-8e46-c4ce4bf40eb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs"
Jul 17 10:03:43.434186 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.434146    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb555f53-125f-41ac-8e46-c4ce4bf40eb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-43c677b2-fznxs\" (UID: \"bb555f53-125f-41ac-8e46-c4ce4bf40eb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs"
Jul 17 10:03:43.434845 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.434823    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb555f53-125f-41ac-8e46-c4ce4bf40eb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-43c677b2-fznxs\" (UID: \"bb555f53-125f-41ac-8e46-c4ce4bf40eb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs"
Jul 17 10:03:43.584005 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.583913    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs"
Jul 17 10:03:43.733077 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.733047    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs"]
Jul 17 10:03:43.734962 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:03:43.734938    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb555f53_125f_41ac_8e46_c4ce4bf40eb9.slice/crio-5998c17cecb264996881f4167fd0491c900413b4fe9b496f89b0271171f2fb83 WatchSource:0}: Error finding container 5998c17cecb264996881f4167fd0491c900413b4fe9b496f89b0271171f2fb83: Status 404 returned error can't find the container with id 5998c17cecb264996881f4167fd0491c900413b4fe9b496f89b0271171f2fb83
Jul 17 10:03:43.760235 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:43.760206    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d9f8c26-e8a7-4843-9ee4-a9b947e71e27" path="/var/lib/kubelet/pods/2d9f8c26-e8a7-4843-9ee4-a9b947e71e27/volumes"
Jul 17 10:03:44.192448 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.192416    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"]
Jul 17 10:03:44.192917 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.192902    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerName="revoke-keys"
Jul 17 10:03:44.192961 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.192918    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d09f287f-4e03-4fd0-8718-59c50f17b9df" containerName="revoke-keys"
Jul 17 10:03:44.193004 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.192995    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b09e6c03-676d-4e2e-b776-c08a596bb2a9" containerName="revoke-keys"
Jul 17 10:03:44.196023 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.196006    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"
Jul 17 10:03:44.203495 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.203448    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"]
Jul 17 10:03:44.211746 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.211703    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs" event={"ID":"bb555f53-125f-41ac-8e46-c4ce4bf40eb9","Type":"ContainerStarted","Data":"1a5dfc413c68d6738351795678e5266aac01569590cd15fe363ef4187b5927a6"}
Jul 17 10:03:44.211919 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.211775    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs" event={"ID":"bb555f53-125f-41ac-8e46-c4ce4bf40eb9","Type":"ContainerStarted","Data":"5998c17cecb264996881f4167fd0491c900413b4fe9b496f89b0271171f2fb83"}
Jul 17 10:03:44.231362 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.231305    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs" podStartSLOduration=1.231288926 podStartE2EDuration="1.231288926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:03:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:03:44.228540943 +0000 UTC m=+2708.293361668" watchObservedRunningTime="2026-07-17 10:03:44.231288926 +0000 UTC m=+2708.296109647"
Jul 17 10:03:44.241229 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.241187    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7fa934c-279a-4de8-b201-e4034c16e5f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk\" (UID: \"a7fa934c-279a-4de8-b201-e4034c16e5f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"
Jul 17 10:03:44.342174 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.342129    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7fa934c-279a-4de8-b201-e4034c16e5f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk\" (UID: \"a7fa934c-279a-4de8-b201-e4034c16e5f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"
Jul 17 10:03:44.342838 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.342818    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7fa934c-279a-4de8-b201-e4034c16e5f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk\" (UID: \"a7fa934c-279a-4de8-b201-e4034c16e5f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"
Jul 17 10:03:44.508912 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.508878    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"
Jul 17 10:03:44.648410 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:44.648385    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"]
Jul 17 10:03:44.650259 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:03:44.650228    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7fa934c_279a_4de8_b201_e4034c16e5f8.slice/crio-be96b97d59545977d3e659a6f620f990f8896a55e1c57e788b20a04a8d5bfb4e WatchSource:0}: Error finding container be96b97d59545977d3e659a6f620f990f8896a55e1c57e788b20a04a8d5bfb4e: Status 404 returned error can't find the container with id be96b97d59545977d3e659a6f620f990f8896a55e1c57e788b20a04a8d5bfb4e
Jul 17 10:03:45.220069 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:45.220038    2571 generic.go:356] "Generic (PLEG): container finished" podID="bb555f53-125f-41ac-8e46-c4ce4bf40eb9" containerID="1a5dfc413c68d6738351795678e5266aac01569590cd15fe363ef4187b5927a6" exitCode=0
Jul 17 10:03:45.220279 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:45.220126    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs" event={"ID":"bb555f53-125f-41ac-8e46-c4ce4bf40eb9","Type":"ContainerDied","Data":"1a5dfc413c68d6738351795678e5266aac01569590cd15fe363ef4187b5927a6"}
Jul 17 10:03:45.221666 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:45.221641    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk" event={"ID":"a7fa934c-279a-4de8-b201-e4034c16e5f8","Type":"ContainerStarted","Data":"6a76a3ea6d4126b65c2d5b522bb4a62cd7d5ff8cfd9a99eca6283f8e23932474"}
Jul 17 10:03:45.221770 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:45.221692    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk" event={"ID":"a7fa934c-279a-4de8-b201-e4034c16e5f8","Type":"ContainerStarted","Data":"be96b97d59545977d3e659a6f620f990f8896a55e1c57e788b20a04a8d5bfb4e"}
Jul 17 10:03:45.256910 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:45.256865    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk" podStartSLOduration=1.256850165 podStartE2EDuration="1.256850165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:03:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:03:45.255968472 +0000 UTC m=+2709.320789207" watchObservedRunningTime="2026-07-17 10:03:45.256850165 +0000 UTC m=+2709.321670886"
Jul 17 10:03:46.228512 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:46.228459    2571 generic.go:356] "Generic (PLEG): container finished" podID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerID="6a76a3ea6d4126b65c2d5b522bb4a62cd7d5ff8cfd9a99eca6283f8e23932474" exitCode=6
Jul 17 10:03:46.228983 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:46.228569    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk" event={"ID":"a7fa934c-279a-4de8-b201-e4034c16e5f8","Type":"ContainerDied","Data":"6a76a3ea6d4126b65c2d5b522bb4a62cd7d5ff8cfd9a99eca6283f8e23932474"}
Jul 17 10:03:46.228983 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:46.228795    2571 scope.go:117] "RemoveContainer" containerID="6a76a3ea6d4126b65c2d5b522bb4a62cd7d5ff8cfd9a99eca6283f8e23932474"
Jul 17 10:03:46.369941 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:46.369914    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs"
Jul 17 10:03:46.460748 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:46.460708    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb555f53-125f-41ac-8e46-c4ce4bf40eb9-maas-api-service-ca\") pod \"bb555f53-125f-41ac-8e46-c4ce4bf40eb9\" (UID: \"bb555f53-125f-41ac-8e46-c4ce4bf40eb9\") "
Jul 17 10:03:46.461071 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:46.461045    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb555f53-125f-41ac-8e46-c4ce4bf40eb9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bb555f53-125f-41ac-8e46-c4ce4bf40eb9" (UID: "bb555f53-125f-41ac-8e46-c4ce4bf40eb9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:03:46.562339 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:46.562294    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb555f53-125f-41ac-8e46-c4ce4bf40eb9-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:03:47.237110 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:47.237071    2571 generic.go:356] "Generic (PLEG): container finished" podID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerID="5174c8cfb9e486280f985fc6eb7083c4cb0fbd7024fd1d0f72cd9d7ad92d0cec" exitCode=6
Jul 17 10:03:47.237589 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:47.237117    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk" event={"ID":"a7fa934c-279a-4de8-b201-e4034c16e5f8","Type":"ContainerDied","Data":"5174c8cfb9e486280f985fc6eb7083c4cb0fbd7024fd1d0f72cd9d7ad92d0cec"}
Jul 17 10:03:47.237589 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:47.237205    2571 scope.go:117] "RemoveContainer" containerID="6a76a3ea6d4126b65c2d5b522bb4a62cd7d5ff8cfd9a99eca6283f8e23932474"
Jul 17 10:03:47.237589 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:47.237401    2571 scope.go:117] "RemoveContainer" containerID="5174c8cfb9e486280f985fc6eb7083c4cb0fbd7024fd1d0f72cd9d7ad92d0cec"
Jul 17 10:03:47.237716 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:03:47.237672    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk_odh-ai-gateway-infra(a7fa934c-279a-4de8-b201-e4034c16e5f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8"
Jul 17 10:03:47.238560 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:47.238541    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs" event={"ID":"bb555f53-125f-41ac-8e46-c4ce4bf40eb9","Type":"ContainerDied","Data":"5998c17cecb264996881f4167fd0491c900413b4fe9b496f89b0271171f2fb83"}
Jul 17 10:03:47.238636 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:47.238579    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5998c17cecb264996881f4167fd0491c900413b4fe9b496f89b0271171f2fb83"
Jul 17 10:03:47.238636 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:47.238580    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-43c677b2-fznxs"
Jul 17 10:03:48.245656 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:03:48.245624    2571 scope.go:117] "RemoveContainer" containerID="5174c8cfb9e486280f985fc6eb7083c4cb0fbd7024fd1d0f72cd9d7ad92d0cec"
Jul 17 10:03:48.246042 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:03:48.245883    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk_odh-ai-gateway-infra(a7fa934c-279a-4de8-b201-e4034c16e5f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8"
Jul 17 10:04:00.752738 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:00.752707    2571 scope.go:117] "RemoveContainer" containerID="5174c8cfb9e486280f985fc6eb7083c4cb0fbd7024fd1d0f72cd9d7ad92d0cec"
Jul 17 10:04:01.323534 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:01.323499    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk" event={"ID":"a7fa934c-279a-4de8-b201-e4034c16e5f8","Type":"ContainerStarted","Data":"206ec99cabc99e205ce7b7dec1a44dd46e7c892261be2dd7836541575b4a254d"}
Jul 17 10:04:02.331262 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:02.331228    2571 generic.go:356] "Generic (PLEG): container finished" podID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerID="206ec99cabc99e205ce7b7dec1a44dd46e7c892261be2dd7836541575b4a254d" exitCode=6
Jul 17 10:04:02.331771 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:02.331299    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk" event={"ID":"a7fa934c-279a-4de8-b201-e4034c16e5f8","Type":"ContainerDied","Data":"206ec99cabc99e205ce7b7dec1a44dd46e7c892261be2dd7836541575b4a254d"}
Jul 17 10:04:02.331771 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:02.331361    2571 scope.go:117] "RemoveContainer" containerID="5174c8cfb9e486280f985fc6eb7083c4cb0fbd7024fd1d0f72cd9d7ad92d0cec"
Jul 17 10:04:02.331771 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:02.331710    2571 scope.go:117] "RemoveContainer" containerID="206ec99cabc99e205ce7b7dec1a44dd46e7c892261be2dd7836541575b4a254d"
Jul 17 10:04:02.331996 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:02.331974    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk_odh-ai-gateway-infra(a7fa934c-279a-4de8-b201-e4034c16e5f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8"
Jul 17 10:04:02.360245 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:02.360214    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"]
Jul 17 10:04:03.117642 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.117603    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"]
Jul 17 10:04:03.118050 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.118038    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb555f53-125f-41ac-8e46-c4ce4bf40eb9" containerName="revoke-keys"
Jul 17 10:04:03.118097 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.118052    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb555f53-125f-41ac-8e46-c4ce4bf40eb9" containerName="revoke-keys"
Jul 17 10:04:03.118134 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.118127    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb555f53-125f-41ac-8e46-c4ce4bf40eb9" containerName="revoke-keys"
Jul 17 10:04:03.122329 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.122312    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"
Jul 17 10:04:03.130982 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.130624    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"]
Jul 17 10:04:03.215519 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.215483    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1d10fc-69e7-4ab5-9e1f-178d43e00850-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g\" (UID: \"bb1d10fc-69e7-4ab5-9e1f-178d43e00850\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"
Jul 17 10:04:03.316944 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.316902    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1d10fc-69e7-4ab5-9e1f-178d43e00850-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g\" (UID: \"bb1d10fc-69e7-4ab5-9e1f-178d43e00850\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"
Jul 17 10:04:03.317592 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.317574    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1d10fc-69e7-4ab5-9e1f-178d43e00850-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g\" (UID: \"bb1d10fc-69e7-4ab5-9e1f-178d43e00850\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"
Jul 17 10:04:03.434914 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.434881    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"
Jul 17 10:04:03.500492 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.500185    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"
Jul 17 10:04:03.581924 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.581892    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"]
Jul 17 10:04:03.584558 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:04:03.584528    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb1d10fc_69e7_4ab5_9e1f_178d43e00850.slice/crio-782bf9f02227316bafc08e0f74c82f3b97da12730c7b91e1241704ee018ad22e WatchSource:0}: Error finding container 782bf9f02227316bafc08e0f74c82f3b97da12730c7b91e1241704ee018ad22e: Status 404 returned error can't find the container with id 782bf9f02227316bafc08e0f74c82f3b97da12730c7b91e1241704ee018ad22e
Jul 17 10:04:03.620397 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.620375    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7fa934c-279a-4de8-b201-e4034c16e5f8-maas-api-service-ca\") pod \"a7fa934c-279a-4de8-b201-e4034c16e5f8\" (UID: \"a7fa934c-279a-4de8-b201-e4034c16e5f8\") "
Jul 17 10:04:03.620751 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.620732    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7fa934c-279a-4de8-b201-e4034c16e5f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a7fa934c-279a-4de8-b201-e4034c16e5f8" (UID: "a7fa934c-279a-4de8-b201-e4034c16e5f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:04:03.721707 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:03.721666    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7fa934c-279a-4de8-b201-e4034c16e5f8-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:04:04.347501 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:04.347453    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"
Jul 17 10:04:04.347697 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:04.347505    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk" event={"ID":"a7fa934c-279a-4de8-b201-e4034c16e5f8","Type":"ContainerDied","Data":"be96b97d59545977d3e659a6f620f990f8896a55e1c57e788b20a04a8d5bfb4e"}
Jul 17 10:04:04.347697 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:04.347569    2571 scope.go:117] "RemoveContainer" containerID="206ec99cabc99e205ce7b7dec1a44dd46e7c892261be2dd7836541575b4a254d"
Jul 17 10:04:04.349286 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:04.349259    2571 generic.go:356] "Generic (PLEG): container finished" podID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerID="66d6b2b71e2d16f82beac6d082d705ea06385d8c6f7c7f27ec9529c8eb3602e6" exitCode=6
Jul 17 10:04:04.349441 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:04.349310    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g" event={"ID":"bb1d10fc-69e7-4ab5-9e1f-178d43e00850","Type":"ContainerDied","Data":"66d6b2b71e2d16f82beac6d082d705ea06385d8c6f7c7f27ec9529c8eb3602e6"}
Jul 17 10:04:04.349441 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:04.349352    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g" event={"ID":"bb1d10fc-69e7-4ab5-9e1f-178d43e00850","Type":"ContainerStarted","Data":"782bf9f02227316bafc08e0f74c82f3b97da12730c7b91e1241704ee018ad22e"}
Jul 17 10:04:04.349659 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:04.349595    2571 scope.go:117] "RemoveContainer" containerID="66d6b2b71e2d16f82beac6d082d705ea06385d8c6f7c7f27ec9529c8eb3602e6"
Jul 17 10:04:04.383569 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:04.383549    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"]
Jul 17 10:04:04.389038 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:04.388989    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-k6ztk"]
Jul 17 10:04:05.296100 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.296069    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"]
Jul 17 10:04:05.296535 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.296522    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerName="revoke-keys"
Jul 17 10:04:05.296585 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.296536    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerName="revoke-keys"
Jul 17 10:04:05.296585 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.296552    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerName="revoke-keys"
Jul 17 10:04:05.296585 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.296557    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerName="revoke-keys"
Jul 17 10:04:05.296585 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.296566    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerName="revoke-keys"
Jul 17 10:04:05.296585 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.296571    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerName="revoke-keys"
Jul 17 10:04:05.296738 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.296645    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerName="revoke-keys"
Jul 17 10:04:05.296738 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.296653    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerName="revoke-keys"
Jul 17 10:04:05.296738 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.296660    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8" containerName="revoke-keys"
Jul 17 10:04:05.299791 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.299770    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"
Jul 17 10:04:05.309657 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.309631    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"]
Jul 17 10:04:05.358575 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.358539    2571 generic.go:356] "Generic (PLEG): container finished" podID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerID="553030a6b17b7b4030b6a49db0cc215116cdd273ae3a0178b68cb1b02d5bb851" exitCode=6
Jul 17 10:04:05.358739 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.358591    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g" event={"ID":"bb1d10fc-69e7-4ab5-9e1f-178d43e00850","Type":"ContainerDied","Data":"553030a6b17b7b4030b6a49db0cc215116cdd273ae3a0178b68cb1b02d5bb851"}
Jul 17 10:04:05.358739 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.358637    2571 scope.go:117] "RemoveContainer" containerID="66d6b2b71e2d16f82beac6d082d705ea06385d8c6f7c7f27ec9529c8eb3602e6"
Jul 17 10:04:05.358862 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.358842    2571 scope.go:117] "RemoveContainer" containerID="553030a6b17b7b4030b6a49db0cc215116cdd273ae3a0178b68cb1b02d5bb851"
Jul 17 10:04:05.359097 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:05.359072    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g_odh-ai-gateway-infra(bb1d10fc-69e7-4ab5-9e1f-178d43e00850)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850"
Jul 17 10:04:05.436498 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.436446    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76cf4ad7-9ab2-4455-ba6d-1befcddf48e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg\" (UID: \"76cf4ad7-9ab2-4455-ba6d-1befcddf48e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"
Jul 17 10:04:05.537648 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.537615    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76cf4ad7-9ab2-4455-ba6d-1befcddf48e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg\" (UID: \"76cf4ad7-9ab2-4455-ba6d-1befcddf48e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"
Jul 17 10:04:05.538242 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.538221    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76cf4ad7-9ab2-4455-ba6d-1befcddf48e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg\" (UID: \"76cf4ad7-9ab2-4455-ba6d-1befcddf48e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"
Jul 17 10:04:05.611915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.611833    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"
Jul 17 10:04:05.759357 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:04:05.759328    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76cf4ad7_9ab2_4455_ba6d_1befcddf48e2.slice/crio-5b44c00f94b9b59fdafa73d52cc939ce13cdea267562d7d09d988c6cb46960bc WatchSource:0}: Error finding container 5b44c00f94b9b59fdafa73d52cc939ce13cdea267562d7d09d988c6cb46960bc: Status 404 returned error can't find the container with id 5b44c00f94b9b59fdafa73d52cc939ce13cdea267562d7d09d988c6cb46960bc
Jul 17 10:04:05.760590 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.760565    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7fa934c-279a-4de8-b201-e4034c16e5f8" path="/var/lib/kubelet/pods/a7fa934c-279a-4de8-b201-e4034c16e5f8/volumes"
Jul 17 10:04:05.760932 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:05.760917    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"]
Jul 17 10:04:06.365369 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:06.365274    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg" event={"ID":"76cf4ad7-9ab2-4455-ba6d-1befcddf48e2","Type":"ContainerStarted","Data":"2c4b54ae59c57e468a9ca84ff4a8c2fdcdab63445a6a7e1b7d6b4dd8af42fda6"}
Jul 17 10:04:06.365369 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:06.365331    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg" event={"ID":"76cf4ad7-9ab2-4455-ba6d-1befcddf48e2","Type":"ContainerStarted","Data":"5b44c00f94b9b59fdafa73d52cc939ce13cdea267562d7d09d988c6cb46960bc"}
Jul 17 10:04:06.366963 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:06.366944    2571 scope.go:117] "RemoveContainer" containerID="553030a6b17b7b4030b6a49db0cc215116cdd273ae3a0178b68cb1b02d5bb851"
Jul 17 10:04:06.367109 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:06.367092    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g_odh-ai-gateway-infra(bb1d10fc-69e7-4ab5-9e1f-178d43e00850)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850"
Jul 17 10:04:06.386624 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:06.386580    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg" podStartSLOduration=1.386566991 podStartE2EDuration="1.386566991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:04:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:04:06.383964326 +0000 UTC m=+2730.448785048" watchObservedRunningTime="2026-07-17 10:04:06.386566991 +0000 UTC m=+2730.451387712"
Jul 17 10:04:07.373878 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:07.373842    2571 generic.go:356] "Generic (PLEG): container finished" podID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerID="2c4b54ae59c57e468a9ca84ff4a8c2fdcdab63445a6a7e1b7d6b4dd8af42fda6" exitCode=6
Jul 17 10:04:07.374261 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:07.373934    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg" event={"ID":"76cf4ad7-9ab2-4455-ba6d-1befcddf48e2","Type":"ContainerDied","Data":"2c4b54ae59c57e468a9ca84ff4a8c2fdcdab63445a6a7e1b7d6b4dd8af42fda6"}
Jul 17 10:04:07.374261 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:07.374077    2571 scope.go:117] "RemoveContainer" containerID="2c4b54ae59c57e468a9ca84ff4a8c2fdcdab63445a6a7e1b7d6b4dd8af42fda6"
Jul 17 10:04:08.382173 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:08.382136    2571 generic.go:356] "Generic (PLEG): container finished" podID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerID="d8ab62bb690a1837a11c1745c7484de3e4a1dc6e0456a6037eeec4e75c7b0629" exitCode=6
Jul 17 10:04:08.382658 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:08.382217    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg" event={"ID":"76cf4ad7-9ab2-4455-ba6d-1befcddf48e2","Type":"ContainerDied","Data":"d8ab62bb690a1837a11c1745c7484de3e4a1dc6e0456a6037eeec4e75c7b0629"}
Jul 17 10:04:08.382658 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:08.382273    2571 scope.go:117] "RemoveContainer" containerID="2c4b54ae59c57e468a9ca84ff4a8c2fdcdab63445a6a7e1b7d6b4dd8af42fda6"
Jul 17 10:04:08.382658 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:08.382521    2571 scope.go:117] "RemoveContainer" containerID="d8ab62bb690a1837a11c1745c7484de3e4a1dc6e0456a6037eeec4e75c7b0629"
Jul 17 10:04:08.382795 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:08.382717    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg_odh-ai-gateway-infra(76cf4ad7-9ab2-4455-ba6d-1befcddf48e2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2"
Jul 17 10:04:09.389457 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:09.389431    2571 scope.go:117] "RemoveContainer" containerID="d8ab62bb690a1837a11c1745c7484de3e4a1dc6e0456a6037eeec4e75c7b0629"
Jul 17 10:04:09.389832 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:09.389635    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg_odh-ai-gateway-infra(76cf4ad7-9ab2-4455-ba6d-1befcddf48e2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2"
Jul 17 10:04:15.580942 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:15.580899    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"]
Jul 17 10:04:15.584779 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:15.584756    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"
Jul 17 10:04:15.593213 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:15.593027    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"]
Jul 17 10:04:15.626795 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:15.626759    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9c585c-b39a-424a-ad78-85641239f31b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn\" (UID: \"1b9c585c-b39a-424a-ad78-85641239f31b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"
Jul 17 10:04:15.727745 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:15.727698    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9c585c-b39a-424a-ad78-85641239f31b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn\" (UID: \"1b9c585c-b39a-424a-ad78-85641239f31b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"
Jul 17 10:04:15.728417 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:15.728394    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9c585c-b39a-424a-ad78-85641239f31b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn\" (UID: \"1b9c585c-b39a-424a-ad78-85641239f31b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"
Jul 17 10:04:15.898248 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:15.898160    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"
Jul 17 10:04:16.048754 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:16.048727    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"]
Jul 17 10:04:16.050263 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:04:16.050235    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9c585c_b39a_424a_ad78_85641239f31b.slice/crio-43da40102c1e1d615554791ff91eab6e0b9f550deaebb8cf4976531b37110b1b WatchSource:0}: Error finding container 43da40102c1e1d615554791ff91eab6e0b9f550deaebb8cf4976531b37110b1b: Status 404 returned error can't find the container with id 43da40102c1e1d615554791ff91eab6e0b9f550deaebb8cf4976531b37110b1b
Jul 17 10:04:16.432315 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:16.432270    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn" event={"ID":"1b9c585c-b39a-424a-ad78-85641239f31b","Type":"ContainerStarted","Data":"c41e0aedf7121848aa1bc854567e22b8415bb535063e3033e4cc982d480e631b"}
Jul 17 10:04:16.432518 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:16.432352    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn" event={"ID":"1b9c585c-b39a-424a-ad78-85641239f31b","Type":"ContainerStarted","Data":"43da40102c1e1d615554791ff91eab6e0b9f550deaebb8cf4976531b37110b1b"}
Jul 17 10:04:17.440368 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:17.440329    2571 generic.go:356] "Generic (PLEG): container finished" podID="1b9c585c-b39a-424a-ad78-85641239f31b" containerID="c41e0aedf7121848aa1bc854567e22b8415bb535063e3033e4cc982d480e631b" exitCode=6
Jul 17 10:04:17.440797 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:17.440411    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn" event={"ID":"1b9c585c-b39a-424a-ad78-85641239f31b","Type":"ContainerDied","Data":"c41e0aedf7121848aa1bc854567e22b8415bb535063e3033e4cc982d480e631b"}
Jul 17 10:04:17.440797 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:17.440618    2571 scope.go:117] "RemoveContainer" containerID="c41e0aedf7121848aa1bc854567e22b8415bb535063e3033e4cc982d480e631b"
Jul 17 10:04:17.755118 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:17.755093    2571 scope.go:117] "RemoveContainer" containerID="553030a6b17b7b4030b6a49db0cc215116cdd273ae3a0178b68cb1b02d5bb851"
Jul 17 10:04:18.450679 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:18.450652    2571 generic.go:356] "Generic (PLEG): container finished" podID="1b9c585c-b39a-424a-ad78-85641239f31b" containerID="65b10250d093b552fee01eb9c43a959a570e6eb11c97b11c27b89ffa2ab46301" exitCode=6
Jul 17 10:04:18.451034 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:18.450735    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn" event={"ID":"1b9c585c-b39a-424a-ad78-85641239f31b","Type":"ContainerDied","Data":"65b10250d093b552fee01eb9c43a959a570e6eb11c97b11c27b89ffa2ab46301"}
Jul 17 10:04:18.451034 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:18.450803    2571 scope.go:117] "RemoveContainer" containerID="c41e0aedf7121848aa1bc854567e22b8415bb535063e3033e4cc982d480e631b"
Jul 17 10:04:18.451034 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:18.451000    2571 scope.go:117] "RemoveContainer" containerID="65b10250d093b552fee01eb9c43a959a570e6eb11c97b11c27b89ffa2ab46301"
Jul 17 10:04:18.451284 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:18.451262    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn_odh-ai-gateway-infra(1b9c585c-b39a-424a-ad78-85641239f31b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn" podUID="1b9c585c-b39a-424a-ad78-85641239f31b"
Jul 17 10:04:18.452857 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:18.452836    2571 generic.go:356] "Generic (PLEG): container finished" podID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerID="25b6a1f04d130fcad635a3549d799db5c3f2a619e879a8ee8d6a05c47fa9781d" exitCode=6
Jul 17 10:04:18.452960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:18.452880    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g" event={"ID":"bb1d10fc-69e7-4ab5-9e1f-178d43e00850","Type":"ContainerDied","Data":"25b6a1f04d130fcad635a3549d799db5c3f2a619e879a8ee8d6a05c47fa9781d"}
Jul 17 10:04:18.453145 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:18.453132    2571 scope.go:117] "RemoveContainer" containerID="25b6a1f04d130fcad635a3549d799db5c3f2a619e879a8ee8d6a05c47fa9781d"
Jul 17 10:04:18.453299 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:18.453284    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g_odh-ai-gateway-infra(bb1d10fc-69e7-4ab5-9e1f-178d43e00850)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850"
Jul 17 10:04:18.491355 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:18.491334    2571 scope.go:117] "RemoveContainer" containerID="553030a6b17b7b4030b6a49db0cc215116cdd273ae3a0178b68cb1b02d5bb851"
Jul 17 10:04:19.459993 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:19.459966    2571 scope.go:117] "RemoveContainer" containerID="65b10250d093b552fee01eb9c43a959a570e6eb11c97b11c27b89ffa2ab46301"
Jul 17 10:04:19.460395 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:19.460193    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn_odh-ai-gateway-infra(1b9c585c-b39a-424a-ad78-85641239f31b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn" podUID="1b9c585c-b39a-424a-ad78-85641239f31b"
Jul 17 10:04:19.499081 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:19.499047    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"]
Jul 17 10:04:19.648307 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:19.648279    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"
Jul 17 10:04:19.667268 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:19.667237    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1d10fc-69e7-4ab5-9e1f-178d43e00850-maas-api-service-ca\") pod \"bb1d10fc-69e7-4ab5-9e1f-178d43e00850\" (UID: \"bb1d10fc-69e7-4ab5-9e1f-178d43e00850\") "
Jul 17 10:04:19.667712 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:19.667680    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb1d10fc-69e7-4ab5-9e1f-178d43e00850-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bb1d10fc-69e7-4ab5-9e1f-178d43e00850" (UID: "bb1d10fc-69e7-4ab5-9e1f-178d43e00850"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:04:19.768310 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:19.768274    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb1d10fc-69e7-4ab5-9e1f-178d43e00850-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:04:20.468888 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:20.468861    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"
Jul 17 10:04:20.468888 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:20.468876    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g" event={"ID":"bb1d10fc-69e7-4ab5-9e1f-178d43e00850","Type":"ContainerDied","Data":"782bf9f02227316bafc08e0f74c82f3b97da12730c7b91e1241704ee018ad22e"}
Jul 17 10:04:20.469392 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:20.468938    2571 scope.go:117] "RemoveContainer" containerID="25b6a1f04d130fcad635a3549d799db5c3f2a619e879a8ee8d6a05c47fa9781d"
Jul 17 10:04:20.494050 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:20.494029    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"]
Jul 17 10:04:20.496726 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:20.496704    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-72k4g"]
Jul 17 10:04:20.752707 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:20.752630    2571 scope.go:117] "RemoveContainer" containerID="d8ab62bb690a1837a11c1745c7484de3e4a1dc6e0456a6037eeec4e75c7b0629"
Jul 17 10:04:21.476987 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:21.476948    2571 generic.go:356] "Generic (PLEG): container finished" podID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerID="231e9ae86d510a3ea024062862727fe3f2887de875ba36f06f7c505ab192bf64" exitCode=6
Jul 17 10:04:21.477445 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:21.477022    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg" event={"ID":"76cf4ad7-9ab2-4455-ba6d-1befcddf48e2","Type":"ContainerDied","Data":"231e9ae86d510a3ea024062862727fe3f2887de875ba36f06f7c505ab192bf64"}
Jul 17 10:04:21.477445 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:21.477076    2571 scope.go:117] "RemoveContainer" containerID="d8ab62bb690a1837a11c1745c7484de3e4a1dc6e0456a6037eeec4e75c7b0629"
Jul 17 10:04:21.477576 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:21.477444    2571 scope.go:117] "RemoveContainer" containerID="231e9ae86d510a3ea024062862727fe3f2887de875ba36f06f7c505ab192bf64"
Jul 17 10:04:21.477731 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:21.477708    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg_odh-ai-gateway-infra(76cf4ad7-9ab2-4455-ba6d-1befcddf48e2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2"
Jul 17 10:04:21.758972 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:21.758938    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" path="/var/lib/kubelet/pods/bb1d10fc-69e7-4ab5-9e1f-178d43e00850/volumes"
Jul 17 10:04:22.515871 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:22.515836    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"]
Jul 17 10:04:22.658071 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:22.658047    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"
Jul 17 10:04:22.695891 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:22.695861    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76cf4ad7-9ab2-4455-ba6d-1befcddf48e2-maas-api-service-ca\") pod \"76cf4ad7-9ab2-4455-ba6d-1befcddf48e2\" (UID: \"76cf4ad7-9ab2-4455-ba6d-1befcddf48e2\") "
Jul 17 10:04:22.696222 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:22.696199    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76cf4ad7-9ab2-4455-ba6d-1befcddf48e2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" (UID: "76cf4ad7-9ab2-4455-ba6d-1befcddf48e2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:04:22.797343 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:22.797256    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76cf4ad7-9ab2-4455-ba6d-1befcddf48e2-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:04:23.493075 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:23.493043    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"
Jul 17 10:04:23.493267 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:23.493083    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg" event={"ID":"76cf4ad7-9ab2-4455-ba6d-1befcddf48e2","Type":"ContainerDied","Data":"5b44c00f94b9b59fdafa73d52cc939ce13cdea267562d7d09d988c6cb46960bc"}
Jul 17 10:04:23.493267 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:23.493151    2571 scope.go:117] "RemoveContainer" containerID="231e9ae86d510a3ea024062862727fe3f2887de875ba36f06f7c505ab192bf64"
Jul 17 10:04:23.518198 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:23.518169    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"]
Jul 17 10:04:23.523070 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:23.523042    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-fhjqg"]
Jul 17 10:04:23.758761 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:23.758667    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" path="/var/lib/kubelet/pods/76cf4ad7-9ab2-4455-ba6d-1befcddf48e2/volumes"
Jul 17 10:04:33.752102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:33.752066    2571 scope.go:117] "RemoveContainer" containerID="65b10250d093b552fee01eb9c43a959a570e6eb11c97b11c27b89ffa2ab46301"
Jul 17 10:04:34.566147 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:34.566115    2571 generic.go:356] "Generic (PLEG): container finished" podID="1b9c585c-b39a-424a-ad78-85641239f31b" containerID="7f7a5d7b9f84d56e84145d9af892e2785e7eed162ef94fe32f8c482df9c4bd98" exitCode=6
Jul 17 10:04:34.566340 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:34.566190    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn" event={"ID":"1b9c585c-b39a-424a-ad78-85641239f31b","Type":"ContainerDied","Data":"7f7a5d7b9f84d56e84145d9af892e2785e7eed162ef94fe32f8c482df9c4bd98"}
Jul 17 10:04:34.566340 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:34.566262    2571 scope.go:117] "RemoveContainer" containerID="65b10250d093b552fee01eb9c43a959a570e6eb11c97b11c27b89ffa2ab46301"
Jul 17 10:04:34.566554 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:34.566537    2571 scope.go:117] "RemoveContainer" containerID="7f7a5d7b9f84d56e84145d9af892e2785e7eed162ef94fe32f8c482df9c4bd98"
Jul 17 10:04:34.566799 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:34.566778    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn_odh-ai-gateway-infra(1b9c585c-b39a-424a-ad78-85641239f31b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn" podUID="1b9c585c-b39a-424a-ad78-85641239f31b"
Jul 17 10:04:35.597003 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:35.596968    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"]
Jul 17 10:04:35.732069 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:35.732044    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"
Jul 17 10:04:35.817557 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:35.817527    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9c585c-b39a-424a-ad78-85641239f31b-maas-api-service-ca\") pod \"1b9c585c-b39a-424a-ad78-85641239f31b\" (UID: \"1b9c585c-b39a-424a-ad78-85641239f31b\") "
Jul 17 10:04:35.817940 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:35.817916    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b9c585c-b39a-424a-ad78-85641239f31b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1b9c585c-b39a-424a-ad78-85641239f31b" (UID: "1b9c585c-b39a-424a-ad78-85641239f31b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:04:35.918548 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:35.918435    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9c585c-b39a-424a-ad78-85641239f31b-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:04:36.581030 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:36.580999    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"
Jul 17 10:04:36.581221 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:36.581031    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn" event={"ID":"1b9c585c-b39a-424a-ad78-85641239f31b","Type":"ContainerDied","Data":"43da40102c1e1d615554791ff91eab6e0b9f550deaebb8cf4976531b37110b1b"}
Jul 17 10:04:36.581221 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:36.581100    2571 scope.go:117] "RemoveContainer" containerID="7f7a5d7b9f84d56e84145d9af892e2785e7eed162ef94fe32f8c482df9c4bd98"
Jul 17 10:04:36.604821 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:36.604791    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"]
Jul 17 10:04:36.610126 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:36.610100    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ff6nn"]
Jul 17 10:04:37.760634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:37.760600    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b9c585c-b39a-424a-ad78-85641239f31b" path="/var/lib/kubelet/pods/1b9c585c-b39a-424a-ad78-85641239f31b/volumes"
Jul 17 10:04:44.246340 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246300    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"]
Jul 17 10:04:44.246803 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246788    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerName="revoke-keys"
Jul 17 10:04:44.246851 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246817    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerName="revoke-keys"
Jul 17 10:04:44.246851 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246826    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9c585c-b39a-424a-ad78-85641239f31b" containerName="revoke-keys"
Jul 17 10:04:44.246851 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246832    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9c585c-b39a-424a-ad78-85641239f31b" containerName="revoke-keys"
Jul 17 10:04:44.246851 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246840    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerName="revoke-keys"
Jul 17 10:04:44.246851 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246847    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246854    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246859    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246872    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9c585c-b39a-424a-ad78-85641239f31b" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246877    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9c585c-b39a-424a-ad78-85641239f31b" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246883    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246889    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246894    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9c585c-b39a-424a-ad78-85641239f31b" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246899    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9c585c-b39a-424a-ad78-85641239f31b" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246976    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246984    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9c585c-b39a-424a-ad78-85641239f31b" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246990    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.246996    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerName="revoke-keys"
Jul 17 10:04:44.247001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.247002    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9c585c-b39a-424a-ad78-85641239f31b" containerName="revoke-keys"
Jul 17 10:04:44.247401 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.247010    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerName="revoke-keys"
Jul 17 10:04:44.250006 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.249986    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"
Jul 17 10:04:44.252587 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.252568    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:04:44.258672 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.258631    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"]
Jul 17 10:04:44.393424 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.393380    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68f89df-be8e-4018-8973-19caf34943c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh\" (UID: \"d68f89df-be8e-4018-8973-19caf34943c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"
Jul 17 10:04:44.494270 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.494229    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68f89df-be8e-4018-8973-19caf34943c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh\" (UID: \"d68f89df-be8e-4018-8973-19caf34943c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"
Jul 17 10:04:44.494922 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.494898    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68f89df-be8e-4018-8973-19caf34943c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh\" (UID: \"d68f89df-be8e-4018-8973-19caf34943c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"
Jul 17 10:04:44.562068 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.561982    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"
Jul 17 10:04:44.911845 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:44.911819    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"]
Jul 17 10:04:44.913000 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:04:44.912971    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68f89df_be8e_4018_8973_19caf34943c1.slice/crio-94f6be25b1f191af6403537667597a71d8b4fd187d1c9c3411a403816ccfc08d WatchSource:0}: Error finding container 94f6be25b1f191af6403537667597a71d8b4fd187d1c9c3411a403816ccfc08d: Status 404 returned error can't find the container with id 94f6be25b1f191af6403537667597a71d8b4fd187d1c9c3411a403816ccfc08d
Jul 17 10:04:45.637881 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:45.637845    2571 generic.go:356] "Generic (PLEG): container finished" podID="d68f89df-be8e-4018-8973-19caf34943c1" containerID="cd2c6cbc8ba3bb9857f5c499c70f869517d677d0be2c2bdc33d10d0a711705db" exitCode=6
Jul 17 10:04:45.638289 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:45.637888    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh" event={"ID":"d68f89df-be8e-4018-8973-19caf34943c1","Type":"ContainerDied","Data":"cd2c6cbc8ba3bb9857f5c499c70f869517d677d0be2c2bdc33d10d0a711705db"}
Jul 17 10:04:45.638289 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:45.637964    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh" event={"ID":"d68f89df-be8e-4018-8973-19caf34943c1","Type":"ContainerStarted","Data":"94f6be25b1f191af6403537667597a71d8b4fd187d1c9c3411a403816ccfc08d"}
Jul 17 10:04:45.638289 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:45.638183    2571 scope.go:117] "RemoveContainer" containerID="cd2c6cbc8ba3bb9857f5c499c70f869517d677d0be2c2bdc33d10d0a711705db"
Jul 17 10:04:46.646137 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:46.646054    2571 generic.go:356] "Generic (PLEG): container finished" podID="d68f89df-be8e-4018-8973-19caf34943c1" containerID="478d1d56162dcf010c0ebc0d41400ec0b94b8057617c8c5a6bf9b521aac474b9" exitCode=6
Jul 17 10:04:46.646137 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:46.646112    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh" event={"ID":"d68f89df-be8e-4018-8973-19caf34943c1","Type":"ContainerDied","Data":"478d1d56162dcf010c0ebc0d41400ec0b94b8057617c8c5a6bf9b521aac474b9"}
Jul 17 10:04:46.646631 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:46.646158    2571 scope.go:117] "RemoveContainer" containerID="cd2c6cbc8ba3bb9857f5c499c70f869517d677d0be2c2bdc33d10d0a711705db"
Jul 17 10:04:46.646631 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:46.646349    2571 scope.go:117] "RemoveContainer" containerID="478d1d56162dcf010c0ebc0d41400ec0b94b8057617c8c5a6bf9b521aac474b9"
Jul 17 10:04:46.646631 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:46.646594    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh_odh-ai-gateway-infra(d68f89df-be8e-4018-8973-19caf34943c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh" podUID="d68f89df-be8e-4018-8973-19caf34943c1"
Jul 17 10:04:47.654888 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:47.654864    2571 scope.go:117] "RemoveContainer" containerID="478d1d56162dcf010c0ebc0d41400ec0b94b8057617c8c5a6bf9b521aac474b9"
Jul 17 10:04:47.655275 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:47.655051    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh_odh-ai-gateway-infra(d68f89df-be8e-4018-8973-19caf34943c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh" podUID="d68f89df-be8e-4018-8973-19caf34943c1"
Jul 17 10:04:55.390676 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.390633    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"]
Jul 17 10:04:55.391259 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.391237    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerName="revoke-keys"
Jul 17 10:04:55.391352 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.391262    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerName="revoke-keys"
Jul 17 10:04:55.391352 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.391310    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerName="revoke-keys"
Jul 17 10:04:55.391352 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.391319    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerName="revoke-keys"
Jul 17 10:04:55.391521 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.391422    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="76cf4ad7-9ab2-4455-ba6d-1befcddf48e2" containerName="revoke-keys"
Jul 17 10:04:55.391521 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.391437    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9c585c-b39a-424a-ad78-85641239f31b" containerName="revoke-keys"
Jul 17 10:04:55.391521 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.391450    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb1d10fc-69e7-4ab5-9e1f-178d43e00850" containerName="revoke-keys"
Jul 17 10:04:55.401077 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.401044    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"
Jul 17 10:04:55.423719 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.423678    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"]
Jul 17 10:04:55.500521 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.500484    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b90a2ee5-1073-4e25-960e-2442217b1eff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2\" (UID: \"b90a2ee5-1073-4e25-960e-2442217b1eff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"
Jul 17 10:04:55.601837 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.601794    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b90a2ee5-1073-4e25-960e-2442217b1eff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2\" (UID: \"b90a2ee5-1073-4e25-960e-2442217b1eff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"
Jul 17 10:04:55.602424 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.602402    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b90a2ee5-1073-4e25-960e-2442217b1eff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2\" (UID: \"b90a2ee5-1073-4e25-960e-2442217b1eff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"
Jul 17 10:04:55.711312 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.711282    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"
Jul 17 10:04:55.875270 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:55.875242    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"]
Jul 17 10:04:55.877129 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:04:55.877101    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb90a2ee5_1073_4e25_960e_2442217b1eff.slice/crio-e098e34051a9789fb9e78a84575785def8e0132d1683dc585da8be89a8b4e158 WatchSource:0}: Error finding container e098e34051a9789fb9e78a84575785def8e0132d1683dc585da8be89a8b4e158: Status 404 returned error can't find the container with id e098e34051a9789fb9e78a84575785def8e0132d1683dc585da8be89a8b4e158
Jul 17 10:04:56.708170 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:56.708135    2571 generic.go:356] "Generic (PLEG): container finished" podID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerID="4b18e422326187d40629b62532dbdf79a4518f3bc55371c1281ef4e175e417e9" exitCode=6
Jul 17 10:04:56.708588 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:56.708183    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2" event={"ID":"b90a2ee5-1073-4e25-960e-2442217b1eff","Type":"ContainerDied","Data":"4b18e422326187d40629b62532dbdf79a4518f3bc55371c1281ef4e175e417e9"}
Jul 17 10:04:56.708588 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:56.708223    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2" event={"ID":"b90a2ee5-1073-4e25-960e-2442217b1eff","Type":"ContainerStarted","Data":"e098e34051a9789fb9e78a84575785def8e0132d1683dc585da8be89a8b4e158"}
Jul 17 10:04:56.708588 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:56.708376    2571 scope.go:117] "RemoveContainer" containerID="4b18e422326187d40629b62532dbdf79a4518f3bc55371c1281ef4e175e417e9"
Jul 17 10:04:57.716073 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:57.716040    2571 generic.go:356] "Generic (PLEG): container finished" podID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerID="a3147a2ef8c7f83cc199a713e339b4be6fa5cb8df25d2fe6c46f222d2a992cbc" exitCode=6
Jul 17 10:04:57.716459 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:57.716096    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2" event={"ID":"b90a2ee5-1073-4e25-960e-2442217b1eff","Type":"ContainerDied","Data":"a3147a2ef8c7f83cc199a713e339b4be6fa5cb8df25d2fe6c46f222d2a992cbc"}
Jul 17 10:04:57.716459 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:57.716142    2571 scope.go:117] "RemoveContainer" containerID="4b18e422326187d40629b62532dbdf79a4518f3bc55371c1281ef4e175e417e9"
Jul 17 10:04:57.716459 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:57.716333    2571 scope.go:117] "RemoveContainer" containerID="a3147a2ef8c7f83cc199a713e339b4be6fa5cb8df25d2fe6c46f222d2a992cbc"
Jul 17 10:04:57.716617 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:57.716583    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2_odh-ai-gateway-infra(b90a2ee5-1073-4e25-960e-2442217b1eff)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff"
Jul 17 10:04:58.724162 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:58.724134    2571 scope.go:117] "RemoveContainer" containerID="a3147a2ef8c7f83cc199a713e339b4be6fa5cb8df25d2fe6c46f222d2a992cbc"
Jul 17 10:04:58.724564 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:04:58.724356    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2_odh-ai-gateway-infra(b90a2ee5-1073-4e25-960e-2442217b1eff)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff"
Jul 17 10:04:59.752160 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:04:59.752124    2571 scope.go:117] "RemoveContainer" containerID="478d1d56162dcf010c0ebc0d41400ec0b94b8057617c8c5a6bf9b521aac474b9"
Jul 17 10:05:00.736975 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:00.736944    2571 generic.go:356] "Generic (PLEG): container finished" podID="d68f89df-be8e-4018-8973-19caf34943c1" containerID="f23ad0bfc585bcb412c75e253ba3b4a317efd114769d1ee09af1e559b6a33a54" exitCode=6
Jul 17 10:05:00.737161 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:00.736985    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh" event={"ID":"d68f89df-be8e-4018-8973-19caf34943c1","Type":"ContainerDied","Data":"f23ad0bfc585bcb412c75e253ba3b4a317efd114769d1ee09af1e559b6a33a54"}
Jul 17 10:05:00.737161 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:00.737079    2571 scope.go:117] "RemoveContainer" containerID="478d1d56162dcf010c0ebc0d41400ec0b94b8057617c8c5a6bf9b521aac474b9"
Jul 17 10:05:00.737400 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:00.737380    2571 scope.go:117] "RemoveContainer" containerID="f23ad0bfc585bcb412c75e253ba3b4a317efd114769d1ee09af1e559b6a33a54"
Jul 17 10:05:00.737631 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:05:00.737607    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh_odh-ai-gateway-infra(d68f89df-be8e-4018-8973-19caf34943c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh" podUID="d68f89df-be8e-4018-8973-19caf34943c1"
Jul 17 10:05:01.797218 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:01.797183    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"]
Jul 17 10:05:01.940358 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:01.940334    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"
Jul 17 10:05:02.059527 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:02.059430    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68f89df-be8e-4018-8973-19caf34943c1-maas-api-service-ca\") pod \"d68f89df-be8e-4018-8973-19caf34943c1\" (UID: \"d68f89df-be8e-4018-8973-19caf34943c1\") "
Jul 17 10:05:02.059784 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:02.059762    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d68f89df-be8e-4018-8973-19caf34943c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d68f89df-be8e-4018-8973-19caf34943c1" (UID: "d68f89df-be8e-4018-8973-19caf34943c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:05:02.160303 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:02.160272    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68f89df-be8e-4018-8973-19caf34943c1-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:05:02.752058 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:02.752022    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh" event={"ID":"d68f89df-be8e-4018-8973-19caf34943c1","Type":"ContainerDied","Data":"94f6be25b1f191af6403537667597a71d8b4fd187d1c9c3411a403816ccfc08d"}
Jul 17 10:05:02.752058 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:02.752047    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"
Jul 17 10:05:02.752317 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:02.752077    2571 scope.go:117] "RemoveContainer" containerID="f23ad0bfc585bcb412c75e253ba3b4a317efd114769d1ee09af1e559b6a33a54"
Jul 17 10:05:02.782431 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:02.782403    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"]
Jul 17 10:05:02.785568 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:02.785543    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-9xmgh"]
Jul 17 10:05:03.758874 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:03.758836    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d68f89df-be8e-4018-8973-19caf34943c1" path="/var/lib/kubelet/pods/d68f89df-be8e-4018-8973-19caf34943c1/volumes"
Jul 17 10:05:10.752395 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:10.752365    2571 scope.go:117] "RemoveContainer" containerID="a3147a2ef8c7f83cc199a713e339b4be6fa5cb8df25d2fe6c46f222d2a992cbc"
Jul 17 10:05:11.805303 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:11.805260    2571 generic.go:356] "Generic (PLEG): container finished" podID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerID="0a4bf596526a163e5efe27370291d81a85a2d199e1ab445e0a359f4b36a662bb" exitCode=6
Jul 17 10:05:11.805820 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:11.805317    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2" event={"ID":"b90a2ee5-1073-4e25-960e-2442217b1eff","Type":"ContainerDied","Data":"0a4bf596526a163e5efe27370291d81a85a2d199e1ab445e0a359f4b36a662bb"}
Jul 17 10:05:11.805820 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:11.805367    2571 scope.go:117] "RemoveContainer" containerID="a3147a2ef8c7f83cc199a713e339b4be6fa5cb8df25d2fe6c46f222d2a992cbc"
Jul 17 10:05:11.805820 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:11.805690    2571 scope.go:117] "RemoveContainer" containerID="0a4bf596526a163e5efe27370291d81a85a2d199e1ab445e0a359f4b36a662bb"
Jul 17 10:05:11.805991 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:05:11.805964    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2_odh-ai-gateway-infra(b90a2ee5-1073-4e25-960e-2442217b1eff)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff"
Jul 17 10:05:12.834953 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:12.834915    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"]
Jul 17 10:05:12.978624 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:12.978598    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"
Jul 17 10:05:13.054162 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:13.054127    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b90a2ee5-1073-4e25-960e-2442217b1eff-maas-api-service-ca\") pod \"b90a2ee5-1073-4e25-960e-2442217b1eff\" (UID: \"b90a2ee5-1073-4e25-960e-2442217b1eff\") "
Jul 17 10:05:13.054509 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:13.054456    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b90a2ee5-1073-4e25-960e-2442217b1eff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b90a2ee5-1073-4e25-960e-2442217b1eff" (UID: "b90a2ee5-1073-4e25-960e-2442217b1eff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:05:13.155774 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:13.155679    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b90a2ee5-1073-4e25-960e-2442217b1eff-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:05:13.819374 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:13.819329    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"
Jul 17 10:05:13.819569 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:13.819328    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2" event={"ID":"b90a2ee5-1073-4e25-960e-2442217b1eff","Type":"ContainerDied","Data":"e098e34051a9789fb9e78a84575785def8e0132d1683dc585da8be89a8b4e158"}
Jul 17 10:05:13.819569 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:13.819494    2571 scope.go:117] "RemoveContainer" containerID="0a4bf596526a163e5efe27370291d81a85a2d199e1ab445e0a359f4b36a662bb"
Jul 17 10:05:13.836896 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:13.836861    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"]
Jul 17 10:05:13.840198 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:13.840172    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-mf7x2"]
Jul 17 10:05:15.758711 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:05:15.758671    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff" path="/var/lib/kubelet/pods/b90a2ee5-1073-4e25-960e-2442217b1eff/volumes"
Jul 17 10:06:05.732754 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.732710    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"]
Jul 17 10:06:05.733347 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733331    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68f89df-be8e-4018-8973-19caf34943c1" containerName="revoke-keys"
Jul 17 10:06:05.733393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733350    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68f89df-be8e-4018-8973-19caf34943c1" containerName="revoke-keys"
Jul 17 10:06:05.733393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733361    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerName="revoke-keys"
Jul 17 10:06:05.733393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733369    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerName="revoke-keys"
Jul 17 10:06:05.733569 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733427    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68f89df-be8e-4018-8973-19caf34943c1" containerName="revoke-keys"
Jul 17 10:06:05.733569 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733436    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68f89df-be8e-4018-8973-19caf34943c1" containerName="revoke-keys"
Jul 17 10:06:05.733569 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733454    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerName="revoke-keys"
Jul 17 10:06:05.733569 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733479    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerName="revoke-keys"
Jul 17 10:06:05.733730 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733581    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerName="revoke-keys"
Jul 17 10:06:05.733730 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733595    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d68f89df-be8e-4018-8973-19caf34943c1" containerName="revoke-keys"
Jul 17 10:06:05.733730 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733605    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d68f89df-be8e-4018-8973-19caf34943c1" containerName="revoke-keys"
Jul 17 10:06:05.733730 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733615    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d68f89df-be8e-4018-8973-19caf34943c1" containerName="revoke-keys"
Jul 17 10:06:05.733730 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.733625    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerName="revoke-keys"
Jul 17 10:06:05.737243 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.737223    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"
Jul 17 10:06:05.741344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.741325    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:06:05.760988 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.760956    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"]
Jul 17 10:06:05.821134 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.821097    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6eee104-a9f7-4238-8430-092e8f9ea89a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv\" (UID: \"c6eee104-a9f7-4238-8430-092e8f9ea89a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"
Jul 17 10:06:05.922276 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.922219    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6eee104-a9f7-4238-8430-092e8f9ea89a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv\" (UID: \"c6eee104-a9f7-4238-8430-092e8f9ea89a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"
Jul 17 10:06:05.923103 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:05.923081    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6eee104-a9f7-4238-8430-092e8f9ea89a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv\" (UID: \"c6eee104-a9f7-4238-8430-092e8f9ea89a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"
Jul 17 10:06:06.048023 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:06.047932    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"
Jul 17 10:06:06.209458 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:06.209427    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"]
Jul 17 10:06:06.209910 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:06:06.209887    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6eee104_a9f7_4238_8430_092e8f9ea89a.slice/crio-b81330d1d964e767545e3a29136e753640f044bb241a1f9bd302fadce1cde450 WatchSource:0}: Error finding container b81330d1d964e767545e3a29136e753640f044bb241a1f9bd302fadce1cde450: Status 404 returned error can't find the container with id b81330d1d964e767545e3a29136e753640f044bb241a1f9bd302fadce1cde450
Jul 17 10:06:07.147474 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:07.147377    2571 generic.go:356] "Generic (PLEG): container finished" podID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerID="00dac5cb0ad64404285d70eddc561cb225e37bc0b17bb87e60ebd98d16c0b697" exitCode=6
Jul 17 10:06:07.147845 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:07.147486    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv" event={"ID":"c6eee104-a9f7-4238-8430-092e8f9ea89a","Type":"ContainerDied","Data":"00dac5cb0ad64404285d70eddc561cb225e37bc0b17bb87e60ebd98d16c0b697"}
Jul 17 10:06:07.147845 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:07.147538    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv" event={"ID":"c6eee104-a9f7-4238-8430-092e8f9ea89a","Type":"ContainerStarted","Data":"b81330d1d964e767545e3a29136e753640f044bb241a1f9bd302fadce1cde450"}
Jul 17 10:06:07.147845 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:07.147659    2571 scope.go:117] "RemoveContainer" containerID="00dac5cb0ad64404285d70eddc561cb225e37bc0b17bb87e60ebd98d16c0b697"
Jul 17 10:06:08.155514 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:08.155413    2571 generic.go:356] "Generic (PLEG): container finished" podID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerID="705575c130ae0c400eb1c82184bc0dcb7b2eefcbb170848cb1c41310623d447a" exitCode=6
Jul 17 10:06:08.155896 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:08.155512    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv" event={"ID":"c6eee104-a9f7-4238-8430-092e8f9ea89a","Type":"ContainerDied","Data":"705575c130ae0c400eb1c82184bc0dcb7b2eefcbb170848cb1c41310623d447a"}
Jul 17 10:06:08.155896 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:08.155578    2571 scope.go:117] "RemoveContainer" containerID="00dac5cb0ad64404285d70eddc561cb225e37bc0b17bb87e60ebd98d16c0b697"
Jul 17 10:06:08.155896 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:08.155814    2571 scope.go:117] "RemoveContainer" containerID="705575c130ae0c400eb1c82184bc0dcb7b2eefcbb170848cb1c41310623d447a"
Jul 17 10:06:08.156073 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:08.156051    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv_odh-ai-gateway-infra(c6eee104-a9f7-4238-8430-092e8f9ea89a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a"
Jul 17 10:06:09.162561 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:09.162528    2571 scope.go:117] "RemoveContainer" containerID="705575c130ae0c400eb1c82184bc0dcb7b2eefcbb170848cb1c41310623d447a"
Jul 17 10:06:09.163035 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:09.162795    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv_odh-ai-gateway-infra(c6eee104-a9f7-4238-8430-092e8f9ea89a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a"
Jul 17 10:06:19.752335 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:19.752305    2571 scope.go:117] "RemoveContainer" containerID="705575c130ae0c400eb1c82184bc0dcb7b2eefcbb170848cb1c41310623d447a"
Jul 17 10:06:20.228941 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:20.228900    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv" event={"ID":"c6eee104-a9f7-4238-8430-092e8f9ea89a","Type":"ContainerStarted","Data":"cd5d407d099e4b5f51de240dcc330188e7f0b9e7edd328e07e48792e5268bf42"}
Jul 17 10:06:20.254894 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:20.254838    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv" podStartSLOduration=15.254819979 podStartE2EDuration="15.254819979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:06:20.25111752 +0000 UTC m=+2864.315938243" watchObservedRunningTime="2026-07-17 10:06:20.254819979 +0000 UTC m=+2864.319640701"
Jul 17 10:06:21.236924 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:21.236891    2571 generic.go:356] "Generic (PLEG): container finished" podID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerID="cd5d407d099e4b5f51de240dcc330188e7f0b9e7edd328e07e48792e5268bf42" exitCode=6
Jul 17 10:06:21.237318 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:21.236963    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv" event={"ID":"c6eee104-a9f7-4238-8430-092e8f9ea89a","Type":"ContainerDied","Data":"cd5d407d099e4b5f51de240dcc330188e7f0b9e7edd328e07e48792e5268bf42"}
Jul 17 10:06:21.237318 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:21.237027    2571 scope.go:117] "RemoveContainer" containerID="705575c130ae0c400eb1c82184bc0dcb7b2eefcbb170848cb1c41310623d447a"
Jul 17 10:06:21.237318 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:21.237295    2571 scope.go:117] "RemoveContainer" containerID="cd5d407d099e4b5f51de240dcc330188e7f0b9e7edd328e07e48792e5268bf42"
Jul 17 10:06:21.237565 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:21.237542    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv_odh-ai-gateway-infra(c6eee104-a9f7-4238-8430-092e8f9ea89a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a"
Jul 17 10:06:21.284216 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:21.284181    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"]
Jul 17 10:06:22.382089 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:22.382064    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"
Jul 17 10:06:22.465291 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:22.465253    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6eee104-a9f7-4238-8430-092e8f9ea89a-maas-api-service-ca\") pod \"c6eee104-a9f7-4238-8430-092e8f9ea89a\" (UID: \"c6eee104-a9f7-4238-8430-092e8f9ea89a\") "
Jul 17 10:06:22.465679 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:22.465658    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6eee104-a9f7-4238-8430-092e8f9ea89a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c6eee104-a9f7-4238-8430-092e8f9ea89a" (UID: "c6eee104-a9f7-4238-8430-092e8f9ea89a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:06:22.566526 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:22.566418    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6eee104-a9f7-4238-8430-092e8f9ea89a-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:06:23.251713 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:23.251684    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"
Jul 17 10:06:23.251894 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:23.251718    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv" event={"ID":"c6eee104-a9f7-4238-8430-092e8f9ea89a","Type":"ContainerDied","Data":"b81330d1d964e767545e3a29136e753640f044bb241a1f9bd302fadce1cde450"}
Jul 17 10:06:23.251894 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:23.251784    2571 scope.go:117] "RemoveContainer" containerID="cd5d407d099e4b5f51de240dcc330188e7f0b9e7edd328e07e48792e5268bf42"
Jul 17 10:06:23.290633 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:23.290600    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"]
Jul 17 10:06:23.293338 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:23.293308    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-kgbfv"]
Jul 17 10:06:23.758658 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:23.758625    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a" path="/var/lib/kubelet/pods/c6eee104-a9f7-4238-8430-092e8f9ea89a/volumes"
Jul 17 10:06:24.376332 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.376285    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"]
Jul 17 10:06:24.376823 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.376805    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerName="revoke-keys"
Jul 17 10:06:24.376915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.376826    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerName="revoke-keys"
Jul 17 10:06:24.376915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.376840    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68f89df-be8e-4018-8973-19caf34943c1" containerName="revoke-keys"
Jul 17 10:06:24.376915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.376848    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68f89df-be8e-4018-8973-19caf34943c1" containerName="revoke-keys"
Jul 17 10:06:24.376915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.376868    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerName="revoke-keys"
Jul 17 10:06:24.376915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.376878    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerName="revoke-keys"
Jul 17 10:06:24.376915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.376894    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerName="revoke-keys"
Jul 17 10:06:24.376915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.376904    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerName="revoke-keys"
Jul 17 10:06:24.377272 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.377020    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b90a2ee5-1073-4e25-960e-2442217b1eff" containerName="revoke-keys"
Jul 17 10:06:24.377272 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.377034    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerName="revoke-keys"
Jul 17 10:06:24.377272 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.377048    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerName="revoke-keys"
Jul 17 10:06:24.381640 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.381616    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"
Jul 17 10:06:24.385668 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.385643    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:06:24.399144 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.399110    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"]
Jul 17 10:06:24.485347 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.485311    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd84ef43-96e1-4df5-b038-fe12f88d6fd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x\" (UID: \"fd84ef43-96e1-4df5-b038-fe12f88d6fd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"
Jul 17 10:06:24.586370 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.586335    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd84ef43-96e1-4df5-b038-fe12f88d6fd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x\" (UID: \"fd84ef43-96e1-4df5-b038-fe12f88d6fd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"
Jul 17 10:06:24.587017 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.586991    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd84ef43-96e1-4df5-b038-fe12f88d6fd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x\" (UID: \"fd84ef43-96e1-4df5-b038-fe12f88d6fd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"
Jul 17 10:06:24.691866 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.691837    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"
Jul 17 10:06:24.840417 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:24.840388    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"]
Jul 17 10:06:24.840916 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:06:24.840894    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd84ef43_96e1_4df5_b038_fe12f88d6fd0.slice/crio-8f6a59a3d98baa5b46cbdb721768711935837322acae9b2e3117249704258cd9 WatchSource:0}: Error finding container 8f6a59a3d98baa5b46cbdb721768711935837322acae9b2e3117249704258cd9: Status 404 returned error can't find the container with id 8f6a59a3d98baa5b46cbdb721768711935837322acae9b2e3117249704258cd9
Jul 17 10:06:25.267948 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:25.267903    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x" event={"ID":"fd84ef43-96e1-4df5-b038-fe12f88d6fd0","Type":"ContainerStarted","Data":"875ed811a6f571a0a83b2dd59e68b7261ccdb3080e343215e1c54366e0ac6f96"}
Jul 17 10:06:25.268123 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:25.268008    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x" event={"ID":"fd84ef43-96e1-4df5-b038-fe12f88d6fd0","Type":"ContainerStarted","Data":"8f6a59a3d98baa5b46cbdb721768711935837322acae9b2e3117249704258cd9"}
Jul 17 10:06:25.288794 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:25.288743    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x" podStartSLOduration=1.288727055 podStartE2EDuration="1.288727055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:06:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:06:25.287346503 +0000 UTC m=+2869.352167226" watchObservedRunningTime="2026-07-17 10:06:25.288727055 +0000 UTC m=+2869.353547827"
Jul 17 10:06:26.275767 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:26.275719    2571 generic.go:356] "Generic (PLEG): container finished" podID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerID="875ed811a6f571a0a83b2dd59e68b7261ccdb3080e343215e1c54366e0ac6f96" exitCode=6
Jul 17 10:06:26.276147 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:26.275818    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x" event={"ID":"fd84ef43-96e1-4df5-b038-fe12f88d6fd0","Type":"ContainerDied","Data":"875ed811a6f571a0a83b2dd59e68b7261ccdb3080e343215e1c54366e0ac6f96"}
Jul 17 10:06:26.276147 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:26.275990    2571 scope.go:117] "RemoveContainer" containerID="875ed811a6f571a0a83b2dd59e68b7261ccdb3080e343215e1c54366e0ac6f96"
Jul 17 10:06:27.284095 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:27.284058    2571 generic.go:356] "Generic (PLEG): container finished" podID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerID="c43469ae2b80b146075848f44dca81a085c0518fad65ddca59202a7f38739e96" exitCode=6
Jul 17 10:06:27.284480 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:27.284140    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x" event={"ID":"fd84ef43-96e1-4df5-b038-fe12f88d6fd0","Type":"ContainerDied","Data":"c43469ae2b80b146075848f44dca81a085c0518fad65ddca59202a7f38739e96"}
Jul 17 10:06:27.284480 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:27.284213    2571 scope.go:117] "RemoveContainer" containerID="875ed811a6f571a0a83b2dd59e68b7261ccdb3080e343215e1c54366e0ac6f96"
Jul 17 10:06:27.284480 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:27.284360    2571 scope.go:117] "RemoveContainer" containerID="c43469ae2b80b146075848f44dca81a085c0518fad65ddca59202a7f38739e96"
Jul 17 10:06:27.284675 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:27.284600    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x_odh-ai-gateway-infra(fd84ef43-96e1-4df5-b038-fe12f88d6fd0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0"
Jul 17 10:06:28.291254 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:28.291174    2571 scope.go:117] "RemoveContainer" containerID="c43469ae2b80b146075848f44dca81a085c0518fad65ddca59202a7f38739e96"
Jul 17 10:06:28.291648 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:28.291404    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x_odh-ai-gateway-infra(fd84ef43-96e1-4df5-b038-fe12f88d6fd0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0"
Jul 17 10:06:35.509876 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:35.509838    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"]
Jul 17 10:06:35.510273 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:35.510261    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerName="revoke-keys"
Jul 17 10:06:35.510332 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:35.510274    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerName="revoke-keys"
Jul 17 10:06:35.510368 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:35.510352    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6eee104-a9f7-4238-8430-092e8f9ea89a" containerName="revoke-keys"
Jul 17 10:06:35.514735 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:35.514718    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"
Jul 17 10:06:35.536197 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:35.536167    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"]
Jul 17 10:06:35.589844 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:35.589804    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25eac7cd-3820-458e-850e-1b8ff54a5467-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn\" (UID: \"25eac7cd-3820-458e-850e-1b8ff54a5467\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"
Jul 17 10:06:35.691197 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:35.691159    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25eac7cd-3820-458e-850e-1b8ff54a5467-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn\" (UID: \"25eac7cd-3820-458e-850e-1b8ff54a5467\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"
Jul 17 10:06:35.691888 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:35.691863    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25eac7cd-3820-458e-850e-1b8ff54a5467-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn\" (UID: \"25eac7cd-3820-458e-850e-1b8ff54a5467\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"
Jul 17 10:06:35.825759 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:35.825682    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"
Jul 17 10:06:35.981320 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:35.981291    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"]
Jul 17 10:06:35.984436 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:06:35.984407    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25eac7cd_3820_458e_850e_1b8ff54a5467.slice/crio-1670113e175dc6864376e425b6f92746a6f8aa91954a370ede17ed39e6dbd14d WatchSource:0}: Error finding container 1670113e175dc6864376e425b6f92746a6f8aa91954a370ede17ed39e6dbd14d: Status 404 returned error can't find the container with id 1670113e175dc6864376e425b6f92746a6f8aa91954a370ede17ed39e6dbd14d
Jul 17 10:06:36.340416 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:36.340374    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn" event={"ID":"25eac7cd-3820-458e-850e-1b8ff54a5467","Type":"ContainerStarted","Data":"6369a0276991902e59b4417b47c261956ee857897aa5e71292453e17854af1ef"}
Jul 17 10:06:36.340639 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:36.340433    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn" event={"ID":"25eac7cd-3820-458e-850e-1b8ff54a5467","Type":"ContainerStarted","Data":"1670113e175dc6864376e425b6f92746a6f8aa91954a370ede17ed39e6dbd14d"}
Jul 17 10:06:36.378884 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:36.378827    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn" podStartSLOduration=1.378811872 podStartE2EDuration="1.378811872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:06:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:06:36.377091011 +0000 UTC m=+2880.441911733" watchObservedRunningTime="2026-07-17 10:06:36.378811872 +0000 UTC m=+2880.443632595"
Jul 17 10:06:37.347496 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:37.347438    2571 generic.go:356] "Generic (PLEG): container finished" podID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerID="6369a0276991902e59b4417b47c261956ee857897aa5e71292453e17854af1ef" exitCode=6
Jul 17 10:06:37.347965 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:37.347507    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn" event={"ID":"25eac7cd-3820-458e-850e-1b8ff54a5467","Type":"ContainerDied","Data":"6369a0276991902e59b4417b47c261956ee857897aa5e71292453e17854af1ef"}
Jul 17 10:06:37.347965 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:37.347792    2571 scope.go:117] "RemoveContainer" containerID="6369a0276991902e59b4417b47c261956ee857897aa5e71292453e17854af1ef"
Jul 17 10:06:38.356440 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:38.356402    2571 generic.go:356] "Generic (PLEG): container finished" podID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerID="e9d0555db6404f3ed9825f768262bb0b7e6a2b6efbbe83b869b70b0c98a8d96b" exitCode=6
Jul 17 10:06:38.356940 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:38.356499    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn" event={"ID":"25eac7cd-3820-458e-850e-1b8ff54a5467","Type":"ContainerDied","Data":"e9d0555db6404f3ed9825f768262bb0b7e6a2b6efbbe83b869b70b0c98a8d96b"}
Jul 17 10:06:38.356940 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:38.356570    2571 scope.go:117] "RemoveContainer" containerID="6369a0276991902e59b4417b47c261956ee857897aa5e71292453e17854af1ef"
Jul 17 10:06:38.356940 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:38.356715    2571 scope.go:117] "RemoveContainer" containerID="e9d0555db6404f3ed9825f768262bb0b7e6a2b6efbbe83b869b70b0c98a8d96b"
Jul 17 10:06:38.357073 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:38.356941    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn_odh-ai-gateway-infra(25eac7cd-3820-458e-850e-1b8ff54a5467)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467"
Jul 17 10:06:39.364222 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:39.364184    2571 scope.go:117] "RemoveContainer" containerID="e9d0555db6404f3ed9825f768262bb0b7e6a2b6efbbe83b869b70b0c98a8d96b"
Jul 17 10:06:39.364634 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:39.364398    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn_odh-ai-gateway-infra(25eac7cd-3820-458e-850e-1b8ff54a5467)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467"
Jul 17 10:06:40.752771 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:40.752738    2571 scope.go:117] "RemoveContainer" containerID="c43469ae2b80b146075848f44dca81a085c0518fad65ddca59202a7f38739e96"
Jul 17 10:06:41.376937 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:41.376907    2571 generic.go:356] "Generic (PLEG): container finished" podID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerID="09483382b72c8ab30471ffa4eb3ef77719946f333f2e45bd99cac0d30d233212" exitCode=6
Jul 17 10:06:41.377061 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:41.376976    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x" event={"ID":"fd84ef43-96e1-4df5-b038-fe12f88d6fd0","Type":"ContainerDied","Data":"09483382b72c8ab30471ffa4eb3ef77719946f333f2e45bd99cac0d30d233212"}
Jul 17 10:06:41.377061 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:41.377035    2571 scope.go:117] "RemoveContainer" containerID="c43469ae2b80b146075848f44dca81a085c0518fad65ddca59202a7f38739e96"
Jul 17 10:06:41.377328 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:41.377309    2571 scope.go:117] "RemoveContainer" containerID="09483382b72c8ab30471ffa4eb3ef77719946f333f2e45bd99cac0d30d233212"
Jul 17 10:06:41.377560 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:41.377542    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x_odh-ai-gateway-infra(fd84ef43-96e1-4df5-b038-fe12f88d6fd0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0"
Jul 17 10:06:42.416223 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:42.416188    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"]
Jul 17 10:06:42.560406 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:42.560379    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"
Jul 17 10:06:42.655768 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:42.655738    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd84ef43-96e1-4df5-b038-fe12f88d6fd0-maas-api-service-ca\") pod \"fd84ef43-96e1-4df5-b038-fe12f88d6fd0\" (UID: \"fd84ef43-96e1-4df5-b038-fe12f88d6fd0\") "
Jul 17 10:06:42.656107 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:42.656086    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd84ef43-96e1-4df5-b038-fe12f88d6fd0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fd84ef43-96e1-4df5-b038-fe12f88d6fd0" (UID: "fd84ef43-96e1-4df5-b038-fe12f88d6fd0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:06:42.756392 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:42.756356    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd84ef43-96e1-4df5-b038-fe12f88d6fd0-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:06:43.391203 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:43.391175    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"
Jul 17 10:06:43.391389 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:43.391225    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x" event={"ID":"fd84ef43-96e1-4df5-b038-fe12f88d6fd0","Type":"ContainerDied","Data":"8f6a59a3d98baa5b46cbdb721768711935837322acae9b2e3117249704258cd9"}
Jul 17 10:06:43.391389 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:43.391286    2571 scope.go:117] "RemoveContainer" containerID="09483382b72c8ab30471ffa4eb3ef77719946f333f2e45bd99cac0d30d233212"
Jul 17 10:06:43.416889 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:43.416856    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"]
Jul 17 10:06:43.421197 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:43.421173    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-tlv7x"]
Jul 17 10:06:43.758388 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:43.758355    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" path="/var/lib/kubelet/pods/fd84ef43-96e1-4df5-b038-fe12f88d6fd0/volumes"
Jul 17 10:06:51.752990 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:51.752952    2571 scope.go:117] "RemoveContainer" containerID="e9d0555db6404f3ed9825f768262bb0b7e6a2b6efbbe83b869b70b0c98a8d96b"
Jul 17 10:06:52.446087 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:52.446054    2571 generic.go:356] "Generic (PLEG): container finished" podID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerID="c8c679a55a4f5f767d11db967465bbdba0cd6d620c1e58246d52f6fa1c2fa654" exitCode=6
Jul 17 10:06:52.446266 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:52.446131    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn" event={"ID":"25eac7cd-3820-458e-850e-1b8ff54a5467","Type":"ContainerDied","Data":"c8c679a55a4f5f767d11db967465bbdba0cd6d620c1e58246d52f6fa1c2fa654"}
Jul 17 10:06:52.446266 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:52.446203    2571 scope.go:117] "RemoveContainer" containerID="e9d0555db6404f3ed9825f768262bb0b7e6a2b6efbbe83b869b70b0c98a8d96b"
Jul 17 10:06:52.446535 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:52.446512    2571 scope.go:117] "RemoveContainer" containerID="c8c679a55a4f5f767d11db967465bbdba0cd6d620c1e58246d52f6fa1c2fa654"
Jul 17 10:06:52.446776 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:52.446752    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn_odh-ai-gateway-infra(25eac7cd-3820-458e-850e-1b8ff54a5467)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467"
Jul 17 10:06:53.422129 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.422092    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"]
Jul 17 10:06:53.422608 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.422532    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerName="revoke-keys"
Jul 17 10:06:53.422608 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.422544    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerName="revoke-keys"
Jul 17 10:06:53.422608 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.422563    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerName="revoke-keys"
Jul 17 10:06:53.422608 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.422568    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerName="revoke-keys"
Jul 17 10:06:53.422608 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.422578    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerName="revoke-keys"
Jul 17 10:06:53.422608 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.422583    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerName="revoke-keys"
Jul 17 10:06:53.422847 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.422648    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerName="revoke-keys"
Jul 17 10:06:53.422847 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.422658    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerName="revoke-keys"
Jul 17 10:06:53.427020 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.427001    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"
Jul 17 10:06:53.455147 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.455109    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"]
Jul 17 10:06:53.534546 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.534512    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"]
Jul 17 10:06:53.554361 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.554308    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7850dcb4-f77b-48e1-be33-e898ccf65aad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9\" (UID: \"7850dcb4-f77b-48e1-be33-e898ccf65aad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"
Jul 17 10:06:53.655887 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.655851    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7850dcb4-f77b-48e1-be33-e898ccf65aad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9\" (UID: \"7850dcb4-f77b-48e1-be33-e898ccf65aad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"
Jul 17 10:06:53.656568 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.656546    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7850dcb4-f77b-48e1-be33-e898ccf65aad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9\" (UID: \"7850dcb4-f77b-48e1-be33-e898ccf65aad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"
Jul 17 10:06:53.681403 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.681341    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"
Jul 17 10:06:53.737772 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.737737    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"
Jul 17 10:06:53.857332 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.857290    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25eac7cd-3820-458e-850e-1b8ff54a5467-maas-api-service-ca\") pod \"25eac7cd-3820-458e-850e-1b8ff54a5467\" (UID: \"25eac7cd-3820-458e-850e-1b8ff54a5467\") "
Jul 17 10:06:53.857798 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.857769    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25eac7cd-3820-458e-850e-1b8ff54a5467-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "25eac7cd-3820-458e-850e-1b8ff54a5467" (UID: "25eac7cd-3820-458e-850e-1b8ff54a5467"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:06:53.937418 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.937386    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"]
Jul 17 10:06:53.939943 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:06:53.939910    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7850dcb4_f77b_48e1_be33_e898ccf65aad.slice/crio-2e6401e739a48f2dd8fbb67e6878dc64305651db122fbb3096ab038b06031fa5 WatchSource:0}: Error finding container 2e6401e739a48f2dd8fbb67e6878dc64305651db122fbb3096ab038b06031fa5: Status 404 returned error can't find the container with id 2e6401e739a48f2dd8fbb67e6878dc64305651db122fbb3096ab038b06031fa5
Jul 17 10:06:53.959167 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:53.959136    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25eac7cd-3820-458e-850e-1b8ff54a5467-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:06:54.462713 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:54.462674    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9" event={"ID":"7850dcb4-f77b-48e1-be33-e898ccf65aad","Type":"ContainerStarted","Data":"d14c89951b44dd61f7cee35add83bfadcc223891b3e0959ce1df3e63c7729d84"}
Jul 17 10:06:54.463148 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:54.462730    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9" event={"ID":"7850dcb4-f77b-48e1-be33-e898ccf65aad","Type":"ContainerStarted","Data":"2e6401e739a48f2dd8fbb67e6878dc64305651db122fbb3096ab038b06031fa5"}
Jul 17 10:06:54.464102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:54.464080    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"
Jul 17 10:06:54.464192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:54.464078    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn" event={"ID":"25eac7cd-3820-458e-850e-1b8ff54a5467","Type":"ContainerDied","Data":"1670113e175dc6864376e425b6f92746a6f8aa91954a370ede17ed39e6dbd14d"}
Jul 17 10:06:54.464246 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:54.464206    2571 scope.go:117] "RemoveContainer" containerID="c8c679a55a4f5f767d11db967465bbdba0cd6d620c1e58246d52f6fa1c2fa654"
Jul 17 10:06:54.495838 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:54.495744    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9" podStartSLOduration=1.495728066 podStartE2EDuration="1.495728066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:06:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:06:54.492601709 +0000 UTC m=+2898.557422431" watchObservedRunningTime="2026-07-17 10:06:54.495728066 +0000 UTC m=+2898.560548787"
Jul 17 10:06:54.519208 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:54.519174    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"]
Jul 17 10:06:54.522518 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:54.522493    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-77zzn"]
Jul 17 10:06:55.471195 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.471161    2571 generic.go:356] "Generic (PLEG): container finished" podID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerID="d14c89951b44dd61f7cee35add83bfadcc223891b3e0959ce1df3e63c7729d84" exitCode=6
Jul 17 10:06:55.471651 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.471247    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9" event={"ID":"7850dcb4-f77b-48e1-be33-e898ccf65aad","Type":"ContainerDied","Data":"d14c89951b44dd61f7cee35add83bfadcc223891b3e0959ce1df3e63c7729d84"}
Jul 17 10:06:55.471651 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.471494    2571 scope.go:117] "RemoveContainer" containerID="d14c89951b44dd61f7cee35add83bfadcc223891b3e0959ce1df3e63c7729d84"
Jul 17 10:06:55.760003 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.759974    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467" path="/var/lib/kubelet/pods/25eac7cd-3820-458e-850e-1b8ff54a5467/volumes"
Jul 17 10:06:55.832712 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.832673    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"]
Jul 17 10:06:55.833324 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.833305    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerName="revoke-keys"
Jul 17 10:06:55.833372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.833329    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerName="revoke-keys"
Jul 17 10:06:55.833372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.833345    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerName="revoke-keys"
Jul 17 10:06:55.833372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.833354    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerName="revoke-keys"
Jul 17 10:06:55.833372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.833365    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerName="revoke-keys"
Jul 17 10:06:55.833526 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.833373    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerName="revoke-keys"
Jul 17 10:06:55.833565 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.833526    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd84ef43-96e1-4df5-b038-fe12f88d6fd0" containerName="revoke-keys"
Jul 17 10:06:55.833565 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.833540    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerName="revoke-keys"
Jul 17 10:06:55.833565 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.833551    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerName="revoke-keys"
Jul 17 10:06:55.833565 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.833561    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="25eac7cd-3820-458e-850e-1b8ff54a5467" containerName="revoke-keys"
Jul 17 10:06:55.837326 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.837296    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"
Jul 17 10:06:55.856126 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.856087    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"]
Jul 17 10:06:55.980817 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:55.980770    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c471db6-8755-473b-8fd2-be16f4741e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh\" (UID: \"2c471db6-8755-473b-8fd2-be16f4741e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"
Jul 17 10:06:56.081437 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:56.081351    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c471db6-8755-473b-8fd2-be16f4741e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh\" (UID: \"2c471db6-8755-473b-8fd2-be16f4741e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"
Jul 17 10:06:56.082059 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:56.082038    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c471db6-8755-473b-8fd2-be16f4741e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh\" (UID: \"2c471db6-8755-473b-8fd2-be16f4741e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"
Jul 17 10:06:56.147709 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:56.147672    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"
Jul 17 10:06:56.316157 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:56.316000    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"]
Jul 17 10:06:56.319219 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:06:56.319195    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c471db6_8755_473b_8fd2_be16f4741e31.slice/crio-54fb1dddc3460806f775354481d8f511ccae6297fcdfda0a26a509c2274a86d6 WatchSource:0}: Error finding container 54fb1dddc3460806f775354481d8f511ccae6297fcdfda0a26a509c2274a86d6: Status 404 returned error can't find the container with id 54fb1dddc3460806f775354481d8f511ccae6297fcdfda0a26a509c2274a86d6
Jul 17 10:06:56.479607 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:56.479569    2571 generic.go:356] "Generic (PLEG): container finished" podID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerID="a63e8745479a248800b758b76511252fc889ed3287e1802365267991824593d0" exitCode=6
Jul 17 10:06:56.480049 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:56.479648    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9" event={"ID":"7850dcb4-f77b-48e1-be33-e898ccf65aad","Type":"ContainerDied","Data":"a63e8745479a248800b758b76511252fc889ed3287e1802365267991824593d0"}
Jul 17 10:06:56.480049 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:56.479715    2571 scope.go:117] "RemoveContainer" containerID="d14c89951b44dd61f7cee35add83bfadcc223891b3e0959ce1df3e63c7729d84"
Jul 17 10:06:56.480049 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:56.479935    2571 scope.go:117] "RemoveContainer" containerID="a63e8745479a248800b758b76511252fc889ed3287e1802365267991824593d0"
Jul 17 10:06:56.480230 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:56.480159    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9_odh-ai-gateway-infra(7850dcb4-f77b-48e1-be33-e898ccf65aad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad"
Jul 17 10:06:56.480936 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:56.480914    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh" event={"ID":"2c471db6-8755-473b-8fd2-be16f4741e31","Type":"ContainerStarted","Data":"54fb1dddc3460806f775354481d8f511ccae6297fcdfda0a26a509c2274a86d6"}
Jul 17 10:06:57.488585 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:57.488552    2571 generic.go:356] "Generic (PLEG): container finished" podID="2c471db6-8755-473b-8fd2-be16f4741e31" containerID="38835c4772558a861d087dd678569802c6d76a6f6b6151d92926fac9290863c7" exitCode=6
Jul 17 10:06:57.489016 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:57.488641    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh" event={"ID":"2c471db6-8755-473b-8fd2-be16f4741e31","Type":"ContainerDied","Data":"38835c4772558a861d087dd678569802c6d76a6f6b6151d92926fac9290863c7"}
Jul 17 10:06:57.489016 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:57.488835    2571 scope.go:117] "RemoveContainer" containerID="38835c4772558a861d087dd678569802c6d76a6f6b6151d92926fac9290863c7"
Jul 17 10:06:57.490619 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:57.490604    2571 scope.go:117] "RemoveContainer" containerID="a63e8745479a248800b758b76511252fc889ed3287e1802365267991824593d0"
Jul 17 10:06:57.490762 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:57.490744    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9_odh-ai-gateway-infra(7850dcb4-f77b-48e1-be33-e898ccf65aad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad"
Jul 17 10:06:58.506034 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:58.506000    2571 generic.go:356] "Generic (PLEG): container finished" podID="2c471db6-8755-473b-8fd2-be16f4741e31" containerID="ac700919cd01bd0e6bf1d73a5ee2480c1e07b751bcb957422cfcc436831fdbde" exitCode=6
Jul 17 10:06:58.506420 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:58.506086    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh" event={"ID":"2c471db6-8755-473b-8fd2-be16f4741e31","Type":"ContainerDied","Data":"ac700919cd01bd0e6bf1d73a5ee2480c1e07b751bcb957422cfcc436831fdbde"}
Jul 17 10:06:58.506420 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:58.506196    2571 scope.go:117] "RemoveContainer" containerID="38835c4772558a861d087dd678569802c6d76a6f6b6151d92926fac9290863c7"
Jul 17 10:06:58.506420 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:58.506414    2571 scope.go:117] "RemoveContainer" containerID="ac700919cd01bd0e6bf1d73a5ee2480c1e07b751bcb957422cfcc436831fdbde"
Jul 17 10:06:58.506663 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:58.506641    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh_odh-ai-gateway-infra(2c471db6-8755-473b-8fd2-be16f4741e31)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh" podUID="2c471db6-8755-473b-8fd2-be16f4741e31"
Jul 17 10:06:59.513234 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:06:59.513206    2571 scope.go:117] "RemoveContainer" containerID="ac700919cd01bd0e6bf1d73a5ee2480c1e07b751bcb957422cfcc436831fdbde"
Jul 17 10:06:59.513714 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:06:59.513507    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh_odh-ai-gateway-infra(2c471db6-8755-473b-8fd2-be16f4741e31)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh" podUID="2c471db6-8755-473b-8fd2-be16f4741e31"
Jul 17 10:07:05.078839 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:05.078799    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb"]
Jul 17 10:07:05.084325 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:05.084300    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb"
Jul 17 10:07:05.091747 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:05.091710    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb"]
Jul 17 10:07:05.159049 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:05.159011    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00699b1c-f31f-4635-8427-353fecff117f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb\" (UID: \"00699b1c-f31f-4635-8427-353fecff117f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb"
Jul 17 10:07:05.260128 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:05.260085    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00699b1c-f31f-4635-8427-353fecff117f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb\" (UID: \"00699b1c-f31f-4635-8427-353fecff117f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb"
Jul 17 10:07:05.260803 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:05.260780    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00699b1c-f31f-4635-8427-353fecff117f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb\" (UID: \"00699b1c-f31f-4635-8427-353fecff117f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb"
Jul 17 10:07:05.397801 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:05.397698    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb"
Jul 17 10:07:05.555432 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:05.555405    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb"]
Jul 17 10:07:05.557748 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:07:05.557718    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00699b1c_f31f_4635_8427_353fecff117f.slice/crio-e438746de359c57c678d4bc2b6d2e6bd1cc3dd100ba078a9419fb1bc1d9630d2 WatchSource:0}: Error finding container e438746de359c57c678d4bc2b6d2e6bd1cc3dd100ba078a9419fb1bc1d9630d2: Status 404 returned error can't find the container with id e438746de359c57c678d4bc2b6d2e6bd1cc3dd100ba078a9419fb1bc1d9630d2
Jul 17 10:07:06.555751 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:06.555660    2571 generic.go:356] "Generic (PLEG): container finished" podID="00699b1c-f31f-4635-8427-353fecff117f" containerID="b474d458f22a4c3901d608a49bae197b106ed33663dab516a9e04607b47616e3" exitCode=0
Jul 17 10:07:06.556132 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:06.555743    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb" event={"ID":"00699b1c-f31f-4635-8427-353fecff117f","Type":"ContainerDied","Data":"b474d458f22a4c3901d608a49bae197b106ed33663dab516a9e04607b47616e3"}
Jul 17 10:07:06.556132 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:06.555808    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb" event={"ID":"00699b1c-f31f-4635-8427-353fecff117f","Type":"ContainerStarted","Data":"e438746de359c57c678d4bc2b6d2e6bd1cc3dd100ba078a9419fb1bc1d9630d2"}
Jul 17 10:07:07.717961 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:07.717937    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb"
Jul 17 10:07:07.884004 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:07.883910    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00699b1c-f31f-4635-8427-353fecff117f-maas-api-service-ca\") pod \"00699b1c-f31f-4635-8427-353fecff117f\" (UID: \"00699b1c-f31f-4635-8427-353fecff117f\") "
Jul 17 10:07:07.884284 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:07.884260    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00699b1c-f31f-4635-8427-353fecff117f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "00699b1c-f31f-4635-8427-353fecff117f" (UID: "00699b1c-f31f-4635-8427-353fecff117f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:07.984694 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:07.984654    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00699b1c-f31f-4635-8427-353fecff117f-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:07:08.569326 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:08.569296    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb"
Jul 17 10:07:08.569549 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:08.569295    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb" event={"ID":"00699b1c-f31f-4635-8427-353fecff117f","Type":"ContainerDied","Data":"e438746de359c57c678d4bc2b6d2e6bd1cc3dd100ba078a9419fb1bc1d9630d2"}
Jul 17 10:07:08.569549 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:08.569432    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e438746de359c57c678d4bc2b6d2e6bd1cc3dd100ba078a9419fb1bc1d9630d2"
Jul 17 10:07:09.752481 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:09.752443    2571 scope.go:117] "RemoveContainer" containerID="a63e8745479a248800b758b76511252fc889ed3287e1802365267991824593d0"
Jul 17 10:07:10.583426 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:10.583393    2571 generic.go:356] "Generic (PLEG): container finished" podID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerID="206dbba67857a64823522bc6b3623104105b027ea1f16040214ee954d9a7b299" exitCode=6
Jul 17 10:07:10.583642 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:10.583479    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9" event={"ID":"7850dcb4-f77b-48e1-be33-e898ccf65aad","Type":"ContainerDied","Data":"206dbba67857a64823522bc6b3623104105b027ea1f16040214ee954d9a7b299"}
Jul 17 10:07:10.583642 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:10.583539    2571 scope.go:117] "RemoveContainer" containerID="a63e8745479a248800b758b76511252fc889ed3287e1802365267991824593d0"
Jul 17 10:07:10.583813 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:10.583798    2571 scope.go:117] "RemoveContainer" containerID="206dbba67857a64823522bc6b3623104105b027ea1f16040214ee954d9a7b299"
Jul 17 10:07:10.584048 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:07:10.584029    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9_odh-ai-gateway-infra(7850dcb4-f77b-48e1-be33-e898ccf65aad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad"
Jul 17 10:07:11.626790 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:11.626748    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"]
Jul 17 10:07:11.778273 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:11.778248    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"
Jul 17 10:07:11.816478 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:11.816425    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7850dcb4-f77b-48e1-be33-e898ccf65aad-maas-api-service-ca\") pod \"7850dcb4-f77b-48e1-be33-e898ccf65aad\" (UID: \"7850dcb4-f77b-48e1-be33-e898ccf65aad\") "
Jul 17 10:07:11.816826 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:11.816795    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7850dcb4-f77b-48e1-be33-e898ccf65aad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7850dcb4-f77b-48e1-be33-e898ccf65aad" (UID: "7850dcb4-f77b-48e1-be33-e898ccf65aad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:11.917346 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:11.917260    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7850dcb4-f77b-48e1-be33-e898ccf65aad-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:07:12.600193 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:12.600163    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"
Jul 17 10:07:12.600350 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:12.600250    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9" event={"ID":"7850dcb4-f77b-48e1-be33-e898ccf65aad","Type":"ContainerDied","Data":"2e6401e739a48f2dd8fbb67e6878dc64305651db122fbb3096ab038b06031fa5"}
Jul 17 10:07:12.600350 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:12.600307    2571 scope.go:117] "RemoveContainer" containerID="206dbba67857a64823522bc6b3623104105b027ea1f16040214ee954d9a7b299"
Jul 17 10:07:12.625472 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:12.625431    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"]
Jul 17 10:07:12.632074 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:12.632047    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-wb5b9"]
Jul 17 10:07:13.759147 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:13.759115    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad" path="/var/lib/kubelet/pods/7850dcb4-f77b-48e1-be33-e898ccf65aad/volumes"
Jul 17 10:07:14.416973 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.416939    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"]
Jul 17 10:07:14.417403 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.417390    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerName="revoke-keys"
Jul 17 10:07:14.417449 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.417405    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerName="revoke-keys"
Jul 17 10:07:14.417449 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.417414    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00699b1c-f31f-4635-8427-353fecff117f" containerName="revoke-keys"
Jul 17 10:07:14.417449 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.417420    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="00699b1c-f31f-4635-8427-353fecff117f" containerName="revoke-keys"
Jul 17 10:07:14.417449 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.417430    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerName="revoke-keys"
Jul 17 10:07:14.417449 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.417436    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerName="revoke-keys"
Jul 17 10:07:14.417634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.417552    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerName="revoke-keys"
Jul 17 10:07:14.417634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.417561    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerName="revoke-keys"
Jul 17 10:07:14.417634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.417573    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="00699b1c-f31f-4635-8427-353fecff117f" containerName="revoke-keys"
Jul 17 10:07:14.422091 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.422074    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"
Jul 17 10:07:14.433605 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.433578    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"]
Jul 17 10:07:14.441211 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.441186    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/936d3106-1d91-4e76-b573-0befc05e3e53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg\" (UID: \"936d3106-1d91-4e76-b573-0befc05e3e53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"
Jul 17 10:07:14.541932 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.541890    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/936d3106-1d91-4e76-b573-0befc05e3e53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg\" (UID: \"936d3106-1d91-4e76-b573-0befc05e3e53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"
Jul 17 10:07:14.542552 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.542534    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/936d3106-1d91-4e76-b573-0befc05e3e53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg\" (UID: \"936d3106-1d91-4e76-b573-0befc05e3e53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"
Jul 17 10:07:14.734155 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.734114    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"
Jul 17 10:07:14.752183 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.752155    2571 scope.go:117] "RemoveContainer" containerID="ac700919cd01bd0e6bf1d73a5ee2480c1e07b751bcb957422cfcc436831fdbde"
Jul 17 10:07:14.884414 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:14.884381    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"]
Jul 17 10:07:14.885706 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:07:14.885681    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod936d3106_1d91_4e76_b573_0befc05e3e53.slice/crio-b51318388eca62bed5de8f80c1350f4cc6f2d6fd2a904bd5737842aff4432b6b WatchSource:0}: Error finding container b51318388eca62bed5de8f80c1350f4cc6f2d6fd2a904bd5737842aff4432b6b: Status 404 returned error can't find the container with id b51318388eca62bed5de8f80c1350f4cc6f2d6fd2a904bd5737842aff4432b6b
Jul 17 10:07:15.628045 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:15.628005    2571 generic.go:356] "Generic (PLEG): container finished" podID="2c471db6-8755-473b-8fd2-be16f4741e31" containerID="13afddfb252f22b9f83f74df6ea0c0b88f5cfb70153a39a39c5f54ee6a26234f" exitCode=6
Jul 17 10:07:15.628290 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:15.628086    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh" event={"ID":"2c471db6-8755-473b-8fd2-be16f4741e31","Type":"ContainerDied","Data":"13afddfb252f22b9f83f74df6ea0c0b88f5cfb70153a39a39c5f54ee6a26234f"}
Jul 17 10:07:15.628290 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:15.628199    2571 scope.go:117] "RemoveContainer" containerID="ac700919cd01bd0e6bf1d73a5ee2480c1e07b751bcb957422cfcc436831fdbde"
Jul 17 10:07:15.628788 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:15.628702    2571 scope.go:117] "RemoveContainer" containerID="13afddfb252f22b9f83f74df6ea0c0b88f5cfb70153a39a39c5f54ee6a26234f"
Jul 17 10:07:15.629088 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:07:15.629047    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh_odh-ai-gateway-infra(2c471db6-8755-473b-8fd2-be16f4741e31)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh" podUID="2c471db6-8755-473b-8fd2-be16f4741e31"
Jul 17 10:07:15.631072 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:15.631045    2571 generic.go:356] "Generic (PLEG): container finished" podID="936d3106-1d91-4e76-b573-0befc05e3e53" containerID="81e061b28a6f40e64547e17ca120bd3438c8fa60b957c90b3c35285cbbcbe4a8" exitCode=6
Jul 17 10:07:15.631188 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:15.631101    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg" event={"ID":"936d3106-1d91-4e76-b573-0befc05e3e53","Type":"ContainerDied","Data":"81e061b28a6f40e64547e17ca120bd3438c8fa60b957c90b3c35285cbbcbe4a8"}
Jul 17 10:07:15.631188 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:15.631142    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg" event={"ID":"936d3106-1d91-4e76-b573-0befc05e3e53","Type":"ContainerStarted","Data":"b51318388eca62bed5de8f80c1350f4cc6f2d6fd2a904bd5737842aff4432b6b"}
Jul 17 10:07:15.631267 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:15.631237    2571 scope.go:117] "RemoveContainer" containerID="81e061b28a6f40e64547e17ca120bd3438c8fa60b957c90b3c35285cbbcbe4a8"
Jul 17 10:07:16.640340 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:16.640309    2571 generic.go:356] "Generic (PLEG): container finished" podID="936d3106-1d91-4e76-b573-0befc05e3e53" containerID="f82aba72fd7280fae7bcb6216c78dc0d28bc6871700499fd38476e64856d3e73" exitCode=6
Jul 17 10:07:16.640799 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:16.640400    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg" event={"ID":"936d3106-1d91-4e76-b573-0befc05e3e53","Type":"ContainerDied","Data":"f82aba72fd7280fae7bcb6216c78dc0d28bc6871700499fd38476e64856d3e73"}
Jul 17 10:07:16.640799 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:16.640498    2571 scope.go:117] "RemoveContainer" containerID="81e061b28a6f40e64547e17ca120bd3438c8fa60b957c90b3c35285cbbcbe4a8"
Jul 17 10:07:16.640799 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:16.640705    2571 scope.go:117] "RemoveContainer" containerID="f82aba72fd7280fae7bcb6216c78dc0d28bc6871700499fd38476e64856d3e73"
Jul 17 10:07:16.640994 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:07:16.640950    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg_odh-ai-gateway-infra(936d3106-1d91-4e76-b573-0befc05e3e53)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg" podUID="936d3106-1d91-4e76-b573-0befc05e3e53"
Jul 17 10:07:16.658155 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:16.658124    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"]
Jul 17 10:07:16.801737 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:16.801705    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"
Jul 17 10:07:16.863769 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:16.863734    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c471db6-8755-473b-8fd2-be16f4741e31-maas-api-service-ca\") pod \"2c471db6-8755-473b-8fd2-be16f4741e31\" (UID: \"2c471db6-8755-473b-8fd2-be16f4741e31\") "
Jul 17 10:07:16.864164 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:16.864141    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c471db6-8755-473b-8fd2-be16f4741e31-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c471db6-8755-473b-8fd2-be16f4741e31" (UID: "2c471db6-8755-473b-8fd2-be16f4741e31"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:16.965323 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:16.965231    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c471db6-8755-473b-8fd2-be16f4741e31-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:07:17.649673 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:17.649647    2571 scope.go:117] "RemoveContainer" containerID="f82aba72fd7280fae7bcb6216c78dc0d28bc6871700499fd38476e64856d3e73"
Jul 17 10:07:17.650115 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:07:17.649903    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg_odh-ai-gateway-infra(936d3106-1d91-4e76-b573-0befc05e3e53)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg" podUID="936d3106-1d91-4e76-b573-0befc05e3e53"
Jul 17 10:07:17.650805 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:17.650784    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"
Jul 17 10:07:17.650921 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:17.650799    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh" event={"ID":"2c471db6-8755-473b-8fd2-be16f4741e31","Type":"ContainerDied","Data":"54fb1dddc3460806f775354481d8f511ccae6297fcdfda0a26a509c2274a86d6"}
Jul 17 10:07:17.650921 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:17.650861    2571 scope.go:117] "RemoveContainer" containerID="13afddfb252f22b9f83f74df6ea0c0b88f5cfb70153a39a39c5f54ee6a26234f"
Jul 17 10:07:17.691193 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:17.691155    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"]
Jul 17 10:07:17.696711 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:17.696682    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-z9vdh"]
Jul 17 10:07:17.759563 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:17.759530    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c471db6-8755-473b-8fd2-be16f4741e31" path="/var/lib/kubelet/pods/2c471db6-8755-473b-8fd2-be16f4741e31/volumes"
Jul 17 10:07:25.562335 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562291    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"]
Jul 17 10:07:25.562837 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562770    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c471db6-8755-473b-8fd2-be16f4741e31" containerName="revoke-keys"
Jul 17 10:07:25.562837 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562781    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c471db6-8755-473b-8fd2-be16f4741e31" containerName="revoke-keys"
Jul 17 10:07:25.562837 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562796    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c471db6-8755-473b-8fd2-be16f4741e31" containerName="revoke-keys"
Jul 17 10:07:25.562837 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562802    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c471db6-8755-473b-8fd2-be16f4741e31" containerName="revoke-keys"
Jul 17 10:07:25.562837 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562811    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c471db6-8755-473b-8fd2-be16f4741e31" containerName="revoke-keys"
Jul 17 10:07:25.562837 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562817    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c471db6-8755-473b-8fd2-be16f4741e31" containerName="revoke-keys"
Jul 17 10:07:25.562837 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562825    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerName="revoke-keys"
Jul 17 10:07:25.562837 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562831    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerName="revoke-keys"
Jul 17 10:07:25.563089 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562905    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c471db6-8755-473b-8fd2-be16f4741e31" containerName="revoke-keys"
Jul 17 10:07:25.563089 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562914    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7850dcb4-f77b-48e1-be33-e898ccf65aad" containerName="revoke-keys"
Jul 17 10:07:25.563089 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562921    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c471db6-8755-473b-8fd2-be16f4741e31" containerName="revoke-keys"
Jul 17 10:07:25.563089 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.562929    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c471db6-8755-473b-8fd2-be16f4741e31" containerName="revoke-keys"
Jul 17 10:07:25.567262 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.567242    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"
Jul 17 10:07:25.575265 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.575136    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"]
Jul 17 10:07:25.642369 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.642328    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds\" (UID: \"4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"
Jul 17 10:07:25.743963 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.743918    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds\" (UID: \"4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"
Jul 17 10:07:25.744668 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.744648    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds\" (UID: \"4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"
Jul 17 10:07:25.879954 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:25.879859    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"
Jul 17 10:07:26.027553 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:26.027495    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"]
Jul 17 10:07:26.030041 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:07:26.030015    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb2a5cf_fb4e_46aa_99c4_8b5005fcb5db.slice/crio-ae411487671ebe776e014c2f538ba350cf78894e2e2a7209a6252155ec48152d WatchSource:0}: Error finding container ae411487671ebe776e014c2f538ba350cf78894e2e2a7209a6252155ec48152d: Status 404 returned error can't find the container with id ae411487671ebe776e014c2f538ba350cf78894e2e2a7209a6252155ec48152d
Jul 17 10:07:26.709949 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:26.709900    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds" event={"ID":"4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db","Type":"ContainerStarted","Data":"1afe54bbdc5771a149e869ded5d21ede288a0db9562c82201f3e995911165f51"}
Jul 17 10:07:26.710358 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:26.709977    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds" event={"ID":"4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db","Type":"ContainerStarted","Data":"ae411487671ebe776e014c2f538ba350cf78894e2e2a7209a6252155ec48152d"}
Jul 17 10:07:26.729515 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:26.729440    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds" podStartSLOduration=1.729425537 podStartE2EDuration="1.729425537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:07:26.726549279 +0000 UTC m=+2930.791370005" watchObservedRunningTime="2026-07-17 10:07:26.729425537 +0000 UTC m=+2930.794246335"
Jul 17 10:07:27.717788 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:27.717756    2571 generic.go:356] "Generic (PLEG): container finished" podID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerID="1afe54bbdc5771a149e869ded5d21ede288a0db9562c82201f3e995911165f51" exitCode=6
Jul 17 10:07:27.718188 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:27.717807    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds" event={"ID":"4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db","Type":"ContainerDied","Data":"1afe54bbdc5771a149e869ded5d21ede288a0db9562c82201f3e995911165f51"}
Jul 17 10:07:27.718188 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:27.718066    2571 scope.go:117] "RemoveContainer" containerID="1afe54bbdc5771a149e869ded5d21ede288a0db9562c82201f3e995911165f51"
Jul 17 10:07:28.726438 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:28.726395    2571 generic.go:356] "Generic (PLEG): container finished" podID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerID="a519be102540716d735bce35351c592491a9650b91038c301c1e3795176ea9ac" exitCode=6
Jul 17 10:07:28.726880 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:28.726497    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds" event={"ID":"4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db","Type":"ContainerDied","Data":"a519be102540716d735bce35351c592491a9650b91038c301c1e3795176ea9ac"}
Jul 17 10:07:28.726880 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:28.726558    2571 scope.go:117] "RemoveContainer" containerID="1afe54bbdc5771a149e869ded5d21ede288a0db9562c82201f3e995911165f51"
Jul 17 10:07:28.726880 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:28.726806    2571 scope.go:117] "RemoveContainer" containerID="a519be102540716d735bce35351c592491a9650b91038c301c1e3795176ea9ac"
Jul 17 10:07:28.727096 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:07:28.727071    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds_odh-ai-gateway-infra(4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db"
Jul 17 10:07:29.734687 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:29.734662    2571 scope.go:117] "RemoveContainer" containerID="a519be102540716d735bce35351c592491a9650b91038c301c1e3795176ea9ac"
Jul 17 10:07:29.735081 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:07:29.734872    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds_odh-ai-gateway-infra(4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db"
Jul 17 10:07:30.752821 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:30.752786    2571 scope.go:117] "RemoveContainer" containerID="f82aba72fd7280fae7bcb6216c78dc0d28bc6871700499fd38476e64856d3e73"
Jul 17 10:07:31.460883 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.460847    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9"]
Jul 17 10:07:31.464571 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.464549    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9"
Jul 17 10:07:31.476276 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.476245    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9"]
Jul 17 10:07:31.602215 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.602175    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f300379f-7ce6-48f2-86ed-98bb66d2ca79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-787694-9ktj9\" (UID: \"f300379f-7ce6-48f2-86ed-98bb66d2ca79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9"
Jul 17 10:07:31.703097 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.703065    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f300379f-7ce6-48f2-86ed-98bb66d2ca79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-787694-9ktj9\" (UID: \"f300379f-7ce6-48f2-86ed-98bb66d2ca79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9"
Jul 17 10:07:31.703825 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.703802    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f300379f-7ce6-48f2-86ed-98bb66d2ca79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-787694-9ktj9\" (UID: \"f300379f-7ce6-48f2-86ed-98bb66d2ca79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9"
Jul 17 10:07:31.751021 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.750986    2571 generic.go:356] "Generic (PLEG): container finished" podID="936d3106-1d91-4e76-b573-0befc05e3e53" containerID="967a11925d8127202d6cfeefd723764c191ec75156c22cd70a630551cb753401" exitCode=6
Jul 17 10:07:31.751190 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.751033    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg" event={"ID":"936d3106-1d91-4e76-b573-0befc05e3e53","Type":"ContainerDied","Data":"967a11925d8127202d6cfeefd723764c191ec75156c22cd70a630551cb753401"}
Jul 17 10:07:31.751190 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.751084    2571 scope.go:117] "RemoveContainer" containerID="f82aba72fd7280fae7bcb6216c78dc0d28bc6871700499fd38476e64856d3e73"
Jul 17 10:07:31.751431 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.751409    2571 scope.go:117] "RemoveContainer" containerID="967a11925d8127202d6cfeefd723764c191ec75156c22cd70a630551cb753401"
Jul 17 10:07:31.751702 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:07:31.751671    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg_odh-ai-gateway-infra(936d3106-1d91-4e76-b573-0befc05e3e53)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg" podUID="936d3106-1d91-4e76-b573-0befc05e3e53"
Jul 17 10:07:31.825767 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.825719    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9"
Jul 17 10:07:31.974088 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:31.974053    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9"]
Jul 17 10:07:31.974226 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:07:31.974093    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf300379f_7ce6_48f2_86ed_98bb66d2ca79.slice/crio-a0803c45a86f1873bfd329c7fde633bc1477b0eb5ccdb45e853e45f150a59f67 WatchSource:0}: Error finding container a0803c45a86f1873bfd329c7fde633bc1477b0eb5ccdb45e853e45f150a59f67: Status 404 returned error can't find the container with id a0803c45a86f1873bfd329c7fde633bc1477b0eb5ccdb45e853e45f150a59f67
Jul 17 10:07:32.758219 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:32.758185    2571 generic.go:356] "Generic (PLEG): container finished" podID="f300379f-7ce6-48f2-86ed-98bb66d2ca79" containerID="c4659dd2fce43680f71927f59f514e5cb7b34a6fa33c2b0b5add56a32c2be725" exitCode=0
Jul 17 10:07:32.758413 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:32.758268    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9" event={"ID":"f300379f-7ce6-48f2-86ed-98bb66d2ca79","Type":"ContainerDied","Data":"c4659dd2fce43680f71927f59f514e5cb7b34a6fa33c2b0b5add56a32c2be725"}
Jul 17 10:07:32.758413 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:32.758355    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9" event={"ID":"f300379f-7ce6-48f2-86ed-98bb66d2ca79","Type":"ContainerStarted","Data":"a0803c45a86f1873bfd329c7fde633bc1477b0eb5ccdb45e853e45f150a59f67"}
Jul 17 10:07:32.793927 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:32.793887    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"]
Jul 17 10:07:32.942873 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:32.942843    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"
Jul 17 10:07:33.015564 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:33.015527    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/936d3106-1d91-4e76-b573-0befc05e3e53-maas-api-service-ca\") pod \"936d3106-1d91-4e76-b573-0befc05e3e53\" (UID: \"936d3106-1d91-4e76-b573-0befc05e3e53\") "
Jul 17 10:07:33.015933 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:33.015908    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/936d3106-1d91-4e76-b573-0befc05e3e53-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "936d3106-1d91-4e76-b573-0befc05e3e53" (UID: "936d3106-1d91-4e76-b573-0befc05e3e53"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:33.116750 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:33.116711    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/936d3106-1d91-4e76-b573-0befc05e3e53-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:07:33.767750 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:33.767720    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"
Jul 17 10:07:33.767750 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:33.767734    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg" event={"ID":"936d3106-1d91-4e76-b573-0befc05e3e53","Type":"ContainerDied","Data":"b51318388eca62bed5de8f80c1350f4cc6f2d6fd2a904bd5737842aff4432b6b"}
Jul 17 10:07:33.767941 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:33.767823    2571 scope.go:117] "RemoveContainer" containerID="967a11925d8127202d6cfeefd723764c191ec75156c22cd70a630551cb753401"
Jul 17 10:07:33.788093 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:33.788062    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"]
Jul 17 10:07:33.792545 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:33.792515    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-7nwjg"]
Jul 17 10:07:33.921411 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:33.921376    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9"
Jul 17 10:07:34.026755 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:34.026654    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f300379f-7ce6-48f2-86ed-98bb66d2ca79-maas-api-service-ca\") pod \"f300379f-7ce6-48f2-86ed-98bb66d2ca79\" (UID: \"f300379f-7ce6-48f2-86ed-98bb66d2ca79\") "
Jul 17 10:07:34.027142 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:34.027032    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f300379f-7ce6-48f2-86ed-98bb66d2ca79-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f300379f-7ce6-48f2-86ed-98bb66d2ca79" (UID: "f300379f-7ce6-48f2-86ed-98bb66d2ca79"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:34.128218 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:34.128177    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f300379f-7ce6-48f2-86ed-98bb66d2ca79-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:07:34.775897 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:34.775868    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9"
Jul 17 10:07:34.776083 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:34.775868    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-787694-9ktj9" event={"ID":"f300379f-7ce6-48f2-86ed-98bb66d2ca79","Type":"ContainerDied","Data":"a0803c45a86f1873bfd329c7fde633bc1477b0eb5ccdb45e853e45f150a59f67"}
Jul 17 10:07:34.776083 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:34.775994    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0803c45a86f1873bfd329c7fde633bc1477b0eb5ccdb45e853e45f150a59f67"
Jul 17 10:07:35.758342 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:35.758304    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="936d3106-1d91-4e76-b573-0befc05e3e53" path="/var/lib/kubelet/pods/936d3106-1d91-4e76-b573-0befc05e3e53/volumes"
Jul 17 10:07:40.752777 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:40.752737    2571 scope.go:117] "RemoveContainer" containerID="a519be102540716d735bce35351c592491a9650b91038c301c1e3795176ea9ac"
Jul 17 10:07:41.818858 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:41.818766    2571 generic.go:356] "Generic (PLEG): container finished" podID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerID="f2d6882819f6615070a2416510da9659586d0db4668d495ab60fca27cf833f3e" exitCode=6
Jul 17 10:07:41.818858 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:41.818838    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds" event={"ID":"4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db","Type":"ContainerDied","Data":"f2d6882819f6615070a2416510da9659586d0db4668d495ab60fca27cf833f3e"}
Jul 17 10:07:41.819411 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:41.818902    2571 scope.go:117] "RemoveContainer" containerID="a519be102540716d735bce35351c592491a9650b91038c301c1e3795176ea9ac"
Jul 17 10:07:41.819411 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:41.819263    2571 scope.go:117] "RemoveContainer" containerID="f2d6882819f6615070a2416510da9659586d0db4668d495ab60fca27cf833f3e"
Jul 17 10:07:41.819565 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:07:41.819537    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds_odh-ai-gateway-infra(4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db"
Jul 17 10:07:42.852414 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:42.852375    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"]
Jul 17 10:07:43.000561 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.000532    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"
Jul 17 10:07:43.107090 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.106996    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db-maas-api-service-ca\") pod \"4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db\" (UID: \"4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db\") "
Jul 17 10:07:43.107347 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.107323    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" (UID: "4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:43.208265 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.208222    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:07:43.425611 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.425531    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"]
Jul 17 10:07:43.425972 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.425961    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="936d3106-1d91-4e76-b573-0befc05e3e53" containerName="revoke-keys"
Jul 17 10:07:43.426014 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.425974    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="936d3106-1d91-4e76-b573-0befc05e3e53" containerName="revoke-keys"
Jul 17 10:07:43.426014 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.425985    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerName="revoke-keys"
Jul 17 10:07:43.426014 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.425990    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerName="revoke-keys"
Jul 17 10:07:43.426014 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.425996    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerName="revoke-keys"
Jul 17 10:07:43.426014 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426003    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerName="revoke-keys"
Jul 17 10:07:43.426014 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426011    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="936d3106-1d91-4e76-b573-0befc05e3e53" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426017    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="936d3106-1d91-4e76-b573-0befc05e3e53" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426022    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="936d3106-1d91-4e76-b573-0befc05e3e53" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426028    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="936d3106-1d91-4e76-b573-0befc05e3e53" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426038    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426044    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426050    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f300379f-7ce6-48f2-86ed-98bb66d2ca79" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426056    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f300379f-7ce6-48f2-86ed-98bb66d2ca79" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426125    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="936d3106-1d91-4e76-b573-0befc05e3e53" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426132    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426137    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="936d3106-1d91-4e76-b573-0befc05e3e53" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426143    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="936d3106-1d91-4e76-b573-0befc05e3e53" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426148    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426152    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f300379f-7ce6-48f2-86ed-98bb66d2ca79" containerName="revoke-keys"
Jul 17 10:07:43.426192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.426158    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" containerName="revoke-keys"
Jul 17 10:07:43.429358 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.429339    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"
Jul 17 10:07:43.438990 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.438956    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"]
Jul 17 10:07:43.511128 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.511087    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e848cdd2-7567-4f33-9233-8c51bc961ff6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj\" (UID: \"e848cdd2-7567-4f33-9233-8c51bc961ff6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"
Jul 17 10:07:43.611855 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.611796    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e848cdd2-7567-4f33-9233-8c51bc961ff6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj\" (UID: \"e848cdd2-7567-4f33-9233-8c51bc961ff6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"
Jul 17 10:07:43.612447 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.612426    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e848cdd2-7567-4f33-9233-8c51bc961ff6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj\" (UID: \"e848cdd2-7567-4f33-9233-8c51bc961ff6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"
Jul 17 10:07:43.741618 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.741580    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"
Jul 17 10:07:43.835373 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.835301    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"
Jul 17 10:07:43.835982 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.835596    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds" event={"ID":"4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db","Type":"ContainerDied","Data":"ae411487671ebe776e014c2f538ba350cf78894e2e2a7209a6252155ec48152d"}
Jul 17 10:07:43.835982 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.835673    2571 scope.go:117] "RemoveContainer" containerID="f2d6882819f6615070a2416510da9659586d0db4668d495ab60fca27cf833f3e"
Jul 17 10:07:43.856931 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.856892    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"]
Jul 17 10:07:43.859923 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.859892    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-m5zds"]
Jul 17 10:07:43.893877 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:43.893846    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"]
Jul 17 10:07:43.895652 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:07:43.895626    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode848cdd2_7567_4f33_9233_8c51bc961ff6.slice/crio-61c9174040abda7bf0e7df50916de304ba91ced5e2369fbfdc9f0f071661d7fc WatchSource:0}: Error finding container 61c9174040abda7bf0e7df50916de304ba91ced5e2369fbfdc9f0f071661d7fc: Status 404 returned error can't find the container with id 61c9174040abda7bf0e7df50916de304ba91ced5e2369fbfdc9f0f071661d7fc
Jul 17 10:07:44.842622 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:44.842582    2571 generic.go:356] "Generic (PLEG): container finished" podID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerID="3e9c185a7f06a4308306859114d40c8eed3ac3c544ca6262b2270b75d80e22a6" exitCode=6
Jul 17 10:07:44.842821 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:44.842672    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj" event={"ID":"e848cdd2-7567-4f33-9233-8c51bc961ff6","Type":"ContainerDied","Data":"3e9c185a7f06a4308306859114d40c8eed3ac3c544ca6262b2270b75d80e22a6"}
Jul 17 10:07:44.842821 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:44.842733    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj" event={"ID":"e848cdd2-7567-4f33-9233-8c51bc961ff6","Type":"ContainerStarted","Data":"61c9174040abda7bf0e7df50916de304ba91ced5e2369fbfdc9f0f071661d7fc"}
Jul 17 10:07:44.842915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:44.842825    2571 scope.go:117] "RemoveContainer" containerID="3e9c185a7f06a4308306859114d40c8eed3ac3c544ca6262b2270b75d80e22a6"
Jul 17 10:07:45.759232 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:45.759198    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db" path="/var/lib/kubelet/pods/4eb2a5cf-fb4e-46aa-99c4-8b5005fcb5db/volumes"
Jul 17 10:07:45.850973 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:45.850888    2571 generic.go:356] "Generic (PLEG): container finished" podID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerID="92245a19b83cdab2f8a41423360149ddac45f0f74d498c453582b7e8c2bd2187" exitCode=6
Jul 17 10:07:45.851117 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:45.850970    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj" event={"ID":"e848cdd2-7567-4f33-9233-8c51bc961ff6","Type":"ContainerDied","Data":"92245a19b83cdab2f8a41423360149ddac45f0f74d498c453582b7e8c2bd2187"}
Jul 17 10:07:45.851117 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:45.851027    2571 scope.go:117] "RemoveContainer" containerID="3e9c185a7f06a4308306859114d40c8eed3ac3c544ca6262b2270b75d80e22a6"
Jul 17 10:07:45.851255 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:45.851241    2571 scope.go:117] "RemoveContainer" containerID="92245a19b83cdab2f8a41423360149ddac45f0f74d498c453582b7e8c2bd2187"
Jul 17 10:07:45.851552 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:07:45.851529    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj_odh-ai-gateway-infra(e848cdd2-7567-4f33-9233-8c51bc961ff6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6"
Jul 17 10:07:46.858807 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:46.858773    2571 scope.go:117] "RemoveContainer" containerID="92245a19b83cdab2f8a41423360149ddac45f0f74d498c453582b7e8c2bd2187"
Jul 17 10:07:46.859221 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:07:46.858983    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj_odh-ai-gateway-infra(e848cdd2-7567-4f33-9233-8c51bc961ff6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6"
Jul 17 10:07:58.752230 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:58.752147    2571 scope.go:117] "RemoveContainer" containerID="92245a19b83cdab2f8a41423360149ddac45f0f74d498c453582b7e8c2bd2187"
Jul 17 10:07:59.939065 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:59.939029    2571 generic.go:356] "Generic (PLEG): container finished" podID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerID="48c348d1070c1abef5316abc4a5281069495364cddff8b93f853d5d81654ba7b" exitCode=6
Jul 17 10:07:59.939540 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:59.939101    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj" event={"ID":"e848cdd2-7567-4f33-9233-8c51bc961ff6","Type":"ContainerDied","Data":"48c348d1070c1abef5316abc4a5281069495364cddff8b93f853d5d81654ba7b"}
Jul 17 10:07:59.939540 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:59.939174    2571 scope.go:117] "RemoveContainer" containerID="92245a19b83cdab2f8a41423360149ddac45f0f74d498c453582b7e8c2bd2187"
Jul 17 10:07:59.939540 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:07:59.939497    2571 scope.go:117] "RemoveContainer" containerID="48c348d1070c1abef5316abc4a5281069495364cddff8b93f853d5d81654ba7b"
Jul 17 10:07:59.939757 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:07:59.939738    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj_odh-ai-gateway-infra(e848cdd2-7567-4f33-9233-8c51bc961ff6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6"
Jul 17 10:08:00.977078 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:00.977046    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"]
Jul 17 10:08:01.147485 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:01.147438    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"
Jul 17 10:08:01.167161 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:01.167139    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e848cdd2-7567-4f33-9233-8c51bc961ff6-maas-api-service-ca\") pod \"e848cdd2-7567-4f33-9233-8c51bc961ff6\" (UID: \"e848cdd2-7567-4f33-9233-8c51bc961ff6\") "
Jul 17 10:08:01.167491 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:01.167435    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e848cdd2-7567-4f33-9233-8c51bc961ff6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e848cdd2-7567-4f33-9233-8c51bc961ff6" (UID: "e848cdd2-7567-4f33-9233-8c51bc961ff6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:01.267791 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:01.267709    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e848cdd2-7567-4f33-9233-8c51bc961ff6-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:08:01.954276 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:01.954246    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"
Jul 17 10:08:01.954441 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:01.954249    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj" event={"ID":"e848cdd2-7567-4f33-9233-8c51bc961ff6","Type":"ContainerDied","Data":"61c9174040abda7bf0e7df50916de304ba91ced5e2369fbfdc9f0f071661d7fc"}
Jul 17 10:08:01.954441 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:01.954383    2571 scope.go:117] "RemoveContainer" containerID="48c348d1070c1abef5316abc4a5281069495364cddff8b93f853d5d81654ba7b"
Jul 17 10:08:01.976910 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:01.976886    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"]
Jul 17 10:08:01.981556 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:01.981533    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-x6xtj"]
Jul 17 10:08:03.758930 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:03.758899    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6" path="/var/lib/kubelet/pods/e848cdd2-7567-4f33-9233-8c51bc961ff6/volumes"
Jul 17 10:08:04.458517 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.458478    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"]
Jul 17 10:08:04.458911 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.458898    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerName="revoke-keys"
Jul 17 10:08:04.458960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.458912    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerName="revoke-keys"
Jul 17 10:08:04.458960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.458923    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerName="revoke-keys"
Jul 17 10:08:04.458960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.458928    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerName="revoke-keys"
Jul 17 10:08:04.459052 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.459002    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerName="revoke-keys"
Jul 17 10:08:04.459052 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.459016    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerName="revoke-keys"
Jul 17 10:08:04.463594 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.463575    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"
Jul 17 10:08:04.466630 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.466600    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:08:04.470147 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.470118    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"]
Jul 17 10:08:04.494360 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.494328    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1892f1a1-f9a6-4457-968f-2f8eac161986-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q\" (UID: \"1892f1a1-f9a6-4457-968f-2f8eac161986\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"
Jul 17 10:08:04.595034 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.594997    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1892f1a1-f9a6-4457-968f-2f8eac161986-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q\" (UID: \"1892f1a1-f9a6-4457-968f-2f8eac161986\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"
Jul 17 10:08:04.595648 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.595627    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1892f1a1-f9a6-4457-968f-2f8eac161986-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q\" (UID: \"1892f1a1-f9a6-4457-968f-2f8eac161986\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"
Jul 17 10:08:04.776414 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:04.776318    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"
Jul 17 10:08:05.132624 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:08:05.132586    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1892f1a1_f9a6_4457_968f_2f8eac161986.slice/crio-7656c2dfb2e3afc675a5334bb652a9fcff5cff5a1b3d06ba6bac8101caf284b8 WatchSource:0}: Error finding container 7656c2dfb2e3afc675a5334bb652a9fcff5cff5a1b3d06ba6bac8101caf284b8: Status 404 returned error can't find the container with id 7656c2dfb2e3afc675a5334bb652a9fcff5cff5a1b3d06ba6bac8101caf284b8
Jul 17 10:08:05.132773 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:05.132616    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"]
Jul 17 10:08:05.980638 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:05.980600    2571 generic.go:356] "Generic (PLEG): container finished" podID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerID="6a8b5347464e8d6f547c94a810a6c80f188489e9f96fa0cb7ab4f2b3a94de9b0" exitCode=6
Jul 17 10:08:05.981035 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:05.980687    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q" event={"ID":"1892f1a1-f9a6-4457-968f-2f8eac161986","Type":"ContainerDied","Data":"6a8b5347464e8d6f547c94a810a6c80f188489e9f96fa0cb7ab4f2b3a94de9b0"}
Jul 17 10:08:05.981035 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:05.980746    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q" event={"ID":"1892f1a1-f9a6-4457-968f-2f8eac161986","Type":"ContainerStarted","Data":"7656c2dfb2e3afc675a5334bb652a9fcff5cff5a1b3d06ba6bac8101caf284b8"}
Jul 17 10:08:05.981035 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:05.980895    2571 scope.go:117] "RemoveContainer" containerID="6a8b5347464e8d6f547c94a810a6c80f188489e9f96fa0cb7ab4f2b3a94de9b0"
Jul 17 10:08:06.988450 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:06.988417    2571 generic.go:356] "Generic (PLEG): container finished" podID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerID="6cc2c90fc17a2faf766de9f85075489c420d4974fdff6fd3b34cc5eb32047b95" exitCode=6
Jul 17 10:08:06.988903 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:06.988505    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q" event={"ID":"1892f1a1-f9a6-4457-968f-2f8eac161986","Type":"ContainerDied","Data":"6cc2c90fc17a2faf766de9f85075489c420d4974fdff6fd3b34cc5eb32047b95"}
Jul 17 10:08:06.988903 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:06.988568    2571 scope.go:117] "RemoveContainer" containerID="6a8b5347464e8d6f547c94a810a6c80f188489e9f96fa0cb7ab4f2b3a94de9b0"
Jul 17 10:08:06.988903 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:06.988808    2571 scope.go:117] "RemoveContainer" containerID="6cc2c90fc17a2faf766de9f85075489c420d4974fdff6fd3b34cc5eb32047b95"
Jul 17 10:08:06.989039 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:06.989017    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q_odh-ai-gateway-infra(1892f1a1-f9a6-4457-968f-2f8eac161986)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986"
Jul 17 10:08:07.996626 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:07.996595    2571 scope.go:117] "RemoveContainer" containerID="6cc2c90fc17a2faf766de9f85075489c420d4974fdff6fd3b34cc5eb32047b95"
Jul 17 10:08:07.997024 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:07.996794    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q_odh-ai-gateway-infra(1892f1a1-f9a6-4457-968f-2f8eac161986)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986"
Jul 17 10:08:15.615863 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:15.615801    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"]
Jul 17 10:08:15.616310 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:15.616297    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerName="revoke-keys"
Jul 17 10:08:15.616354 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:15.616312    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerName="revoke-keys"
Jul 17 10:08:15.616414 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:15.616403    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e848cdd2-7567-4f33-9233-8c51bc961ff6" containerName="revoke-keys"
Jul 17 10:08:15.620829 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:15.620809    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"
Jul 17 10:08:15.633312 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:15.633285    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"]
Jul 17 10:08:15.702933 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:15.702899    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/266493d2-e072-4f39-9040-e72c23950fc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn\" (UID: \"266493d2-e072-4f39-9040-e72c23950fc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"
Jul 17 10:08:15.803902 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:15.803863    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/266493d2-e072-4f39-9040-e72c23950fc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn\" (UID: \"266493d2-e072-4f39-9040-e72c23950fc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"
Jul 17 10:08:15.804560 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:15.804540    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/266493d2-e072-4f39-9040-e72c23950fc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn\" (UID: \"266493d2-e072-4f39-9040-e72c23950fc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"
Jul 17 10:08:15.931740 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:15.931651    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"
Jul 17 10:08:16.324908 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:16.324869    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"]
Jul 17 10:08:16.328671 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:08:16.328642    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266493d2_e072_4f39_9040_e72c23950fc1.slice/crio-c1085a3fea326154f2b1ded907cdd08fc1d80c9b3e947b5d745d373592aea7a9 WatchSource:0}: Error finding container c1085a3fea326154f2b1ded907cdd08fc1d80c9b3e947b5d745d373592aea7a9: Status 404 returned error can't find the container with id c1085a3fea326154f2b1ded907cdd08fc1d80c9b3e947b5d745d373592aea7a9
Jul 17 10:08:17.055613 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:17.055576    2571 generic.go:356] "Generic (PLEG): container finished" podID="266493d2-e072-4f39-9040-e72c23950fc1" containerID="48f516b73064b6113ba6a0199ce3e9c22f11b8925784f5dc5bac2af83d3aca96" exitCode=6
Jul 17 10:08:17.056122 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:17.055671    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn" event={"ID":"266493d2-e072-4f39-9040-e72c23950fc1","Type":"ContainerDied","Data":"48f516b73064b6113ba6a0199ce3e9c22f11b8925784f5dc5bac2af83d3aca96"}
Jul 17 10:08:17.056122 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:17.055715    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn" event={"ID":"266493d2-e072-4f39-9040-e72c23950fc1","Type":"ContainerStarted","Data":"c1085a3fea326154f2b1ded907cdd08fc1d80c9b3e947b5d745d373592aea7a9"}
Jul 17 10:08:17.056122 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:17.055929    2571 scope.go:117] "RemoveContainer" containerID="48f516b73064b6113ba6a0199ce3e9c22f11b8925784f5dc5bac2af83d3aca96"
Jul 17 10:08:18.064099 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:18.064060    2571 generic.go:356] "Generic (PLEG): container finished" podID="266493d2-e072-4f39-9040-e72c23950fc1" containerID="f8e62cb3128e2a00a790e83678ec6561171c7a1778d5cbaf21360f20e0ebb061" exitCode=6
Jul 17 10:08:18.064596 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:18.064148    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn" event={"ID":"266493d2-e072-4f39-9040-e72c23950fc1","Type":"ContainerDied","Data":"f8e62cb3128e2a00a790e83678ec6561171c7a1778d5cbaf21360f20e0ebb061"}
Jul 17 10:08:18.064596 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:18.064211    2571 scope.go:117] "RemoveContainer" containerID="48f516b73064b6113ba6a0199ce3e9c22f11b8925784f5dc5bac2af83d3aca96"
Jul 17 10:08:18.064596 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:18.064503    2571 scope.go:117] "RemoveContainer" containerID="f8e62cb3128e2a00a790e83678ec6561171c7a1778d5cbaf21360f20e0ebb061"
Jul 17 10:08:18.064772 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:18.064752    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn_odh-ai-gateway-infra(266493d2-e072-4f39-9040-e72c23950fc1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn" podUID="266493d2-e072-4f39-9040-e72c23950fc1"
Jul 17 10:08:19.072271 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:19.072244    2571 scope.go:117] "RemoveContainer" containerID="f8e62cb3128e2a00a790e83678ec6561171c7a1778d5cbaf21360f20e0ebb061"
Jul 17 10:08:19.072805 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:19.072529    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn_odh-ai-gateway-infra(266493d2-e072-4f39-9040-e72c23950fc1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn" podUID="266493d2-e072-4f39-9040-e72c23950fc1"
Jul 17 10:08:19.753149 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:19.753114    2571 scope.go:117] "RemoveContainer" containerID="6cc2c90fc17a2faf766de9f85075489c420d4974fdff6fd3b34cc5eb32047b95"
Jul 17 10:08:20.079907 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:20.079877    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q" event={"ID":"1892f1a1-f9a6-4457-968f-2f8eac161986","Type":"ContainerStarted","Data":"14d93729f3f034e0299596b3971344363599980ca1105452378044e42ebc70ea"}
Jul 17 10:08:20.101931 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:20.101880    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q" podStartSLOduration=16.101864333 podStartE2EDuration="16.101864333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:08:20.101575619 +0000 UTC m=+2984.166396340" watchObservedRunningTime="2026-07-17 10:08:20.101864333 +0000 UTC m=+2984.166685073"
Jul 17 10:08:21.087393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:21.087358    2571 generic.go:356] "Generic (PLEG): container finished" podID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerID="14d93729f3f034e0299596b3971344363599980ca1105452378044e42ebc70ea" exitCode=6
Jul 17 10:08:21.087845 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:21.087428    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q" event={"ID":"1892f1a1-f9a6-4457-968f-2f8eac161986","Type":"ContainerDied","Data":"14d93729f3f034e0299596b3971344363599980ca1105452378044e42ebc70ea"}
Jul 17 10:08:21.087845 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:21.087506    2571 scope.go:117] "RemoveContainer" containerID="6cc2c90fc17a2faf766de9f85075489c420d4974fdff6fd3b34cc5eb32047b95"
Jul 17 10:08:21.087845 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:21.087799    2571 scope.go:117] "RemoveContainer" containerID="14d93729f3f034e0299596b3971344363599980ca1105452378044e42ebc70ea"
Jul 17 10:08:21.088079 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:21.088056    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q_odh-ai-gateway-infra(1892f1a1-f9a6-4457-968f-2f8eac161986)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986"
Jul 17 10:08:21.129233 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:21.129203    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"]
Jul 17 10:08:22.240613 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.240586    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"
Jul 17 10:08:22.368987 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.368897    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1892f1a1-f9a6-4457-968f-2f8eac161986-maas-api-service-ca\") pod \"1892f1a1-f9a6-4457-968f-2f8eac161986\" (UID: \"1892f1a1-f9a6-4457-968f-2f8eac161986\") "
Jul 17 10:08:22.369271 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.369249    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1892f1a1-f9a6-4457-968f-2f8eac161986-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1892f1a1-f9a6-4457-968f-2f8eac161986" (UID: "1892f1a1-f9a6-4457-968f-2f8eac161986"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:22.469982 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.469941    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1892f1a1-f9a6-4457-968f-2f8eac161986-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:08:22.992873 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.992836    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz"]
Jul 17 10:08:22.993296 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.993284    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerName="revoke-keys"
Jul 17 10:08:22.993337 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.993298    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerName="revoke-keys"
Jul 17 10:08:22.993337 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.993320    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerName="revoke-keys"
Jul 17 10:08:22.993337 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.993325    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerName="revoke-keys"
Jul 17 10:08:22.993424 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.993401    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerName="revoke-keys"
Jul 17 10:08:22.993424 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.993409    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerName="revoke-keys"
Jul 17 10:08:22.993424 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.993416    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerName="revoke-keys"
Jul 17 10:08:22.996547 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:22.996531    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz"
Jul 17 10:08:23.016376 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.016343    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz"]
Jul 17 10:08:23.075320 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.075275    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d7262f-x24mz\" (UID: \"fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz"
Jul 17 10:08:23.102642 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.102614    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"
Jul 17 10:08:23.102825 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.102684    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q" event={"ID":"1892f1a1-f9a6-4457-968f-2f8eac161986","Type":"ContainerDied","Data":"7656c2dfb2e3afc675a5334bb652a9fcff5cff5a1b3d06ba6bac8101caf284b8"}
Jul 17 10:08:23.102825 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.102751    2571 scope.go:117] "RemoveContainer" containerID="14d93729f3f034e0299596b3971344363599980ca1105452378044e42ebc70ea"
Jul 17 10:08:23.151969 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.151936    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"]
Jul 17 10:08:23.161243 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.161214    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-mrd8q"]
Jul 17 10:08:23.176447 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.176415    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d7262f-x24mz\" (UID: \"fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz"
Jul 17 10:08:23.177206 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.177186    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d7262f-x24mz\" (UID: \"fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz"
Jul 17 10:08:23.307638 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.307536    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz"
Jul 17 10:08:23.465984 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:08:23.465927    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc51fb3_ae26_4fa3_b820_f5a3aca8b07b.slice/crio-4d2b88ab835d9813f42cae61ffccccaf2cb379150c4cdbd12ae3a3b78c24fc5d WatchSource:0}: Error finding container 4d2b88ab835d9813f42cae61ffccccaf2cb379150c4cdbd12ae3a3b78c24fc5d: Status 404 returned error can't find the container with id 4d2b88ab835d9813f42cae61ffccccaf2cb379150c4cdbd12ae3a3b78c24fc5d
Jul 17 10:08:23.466250 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.466225    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz"]
Jul 17 10:08:23.759322 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:23.759288    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986" path="/var/lib/kubelet/pods/1892f1a1-f9a6-4457-968f-2f8eac161986/volumes"
Jul 17 10:08:24.110447 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:24.110413    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz" event={"ID":"fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b","Type":"ContainerStarted","Data":"bcf007e097089166808007218bbc42d31c8a4b85fcd579c74329b823e1df8b2a"}
Jul 17 10:08:24.110635 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:24.110486    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz" event={"ID":"fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b","Type":"ContainerStarted","Data":"4d2b88ab835d9813f42cae61ffccccaf2cb379150c4cdbd12ae3a3b78c24fc5d"}
Jul 17 10:08:24.176268 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:24.176206    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz" podStartSLOduration=2.176188955 podStartE2EDuration="2.176188955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:08:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:08:24.172806503 +0000 UTC m=+2988.237627225" watchObservedRunningTime="2026-07-17 10:08:24.176188955 +0000 UTC m=+2988.241009677"
Jul 17 10:08:25.119418 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:25.119384    2571 generic.go:356] "Generic (PLEG): container finished" podID="fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b" containerID="bcf007e097089166808007218bbc42d31c8a4b85fcd579c74329b823e1df8b2a" exitCode=0
Jul 17 10:08:25.119847 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:25.119438    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz" event={"ID":"fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b","Type":"ContainerDied","Data":"bcf007e097089166808007218bbc42d31c8a4b85fcd579c74329b823e1df8b2a"}
Jul 17 10:08:26.265094 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:26.265068    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz"
Jul 17 10:08:26.409291 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:26.409192    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b-maas-api-service-ca\") pod \"fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b\" (UID: \"fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b\") "
Jul 17 10:08:26.409589 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:26.409566    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b" (UID: "fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:26.511055 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:26.511019    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:08:27.133064 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:27.133030    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz" event={"ID":"fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b","Type":"ContainerDied","Data":"4d2b88ab835d9813f42cae61ffccccaf2cb379150c4cdbd12ae3a3b78c24fc5d"}
Jul 17 10:08:27.133064 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:27.133047    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d7262f-x24mz"
Jul 17 10:08:27.133329 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:27.133077    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d2b88ab835d9813f42cae61ffccccaf2cb379150c4cdbd12ae3a3b78c24fc5d"
Jul 17 10:08:31.753116 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:31.753086    2571 scope.go:117] "RemoveContainer" containerID="f8e62cb3128e2a00a790e83678ec6561171c7a1778d5cbaf21360f20e0ebb061"
Jul 17 10:08:32.165086 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:32.165055    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn" event={"ID":"266493d2-e072-4f39-9040-e72c23950fc1","Type":"ContainerStarted","Data":"74e9b6b2d47362cdb270370b7dc66140ff0d08c079368aaca8a336728a378c16"}
Jul 17 10:08:32.186995 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:32.186940    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn" podStartSLOduration=17.186925055 podStartE2EDuration="17.186925055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:08:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:08:32.183947739 +0000 UTC m=+2996.248768466" watchObservedRunningTime="2026-07-17 10:08:32.186925055 +0000 UTC m=+2996.251745776"
Jul 17 10:08:33.173716 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.173677    2571 generic.go:356] "Generic (PLEG): container finished" podID="266493d2-e072-4f39-9040-e72c23950fc1" containerID="74e9b6b2d47362cdb270370b7dc66140ff0d08c079368aaca8a336728a378c16" exitCode=6
Jul 17 10:08:33.174150 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.173720    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn" event={"ID":"266493d2-e072-4f39-9040-e72c23950fc1","Type":"ContainerDied","Data":"74e9b6b2d47362cdb270370b7dc66140ff0d08c079368aaca8a336728a378c16"}
Jul 17 10:08:33.174150 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.173788    2571 scope.go:117] "RemoveContainer" containerID="f8e62cb3128e2a00a790e83678ec6561171c7a1778d5cbaf21360f20e0ebb061"
Jul 17 10:08:33.174150 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.174045    2571 scope.go:117] "RemoveContainer" containerID="74e9b6b2d47362cdb270370b7dc66140ff0d08c079368aaca8a336728a378c16"
Jul 17 10:08:33.174299 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:33.174238    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn_odh-ai-gateway-infra(266493d2-e072-4f39-9040-e72c23950fc1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn" podUID="266493d2-e072-4f39-9040-e72c23950fc1"
Jul 17 10:08:33.203116 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.203085    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"]
Jul 17 10:08:33.479172 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.479132    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"]
Jul 17 10:08:33.479964 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.479941    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerName="revoke-keys"
Jul 17 10:08:33.480052 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.479967    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1892f1a1-f9a6-4457-968f-2f8eac161986" containerName="revoke-keys"
Jul 17 10:08:33.480052 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.479983    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b" containerName="revoke-keys"
Jul 17 10:08:33.480052 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.479992    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b" containerName="revoke-keys"
Jul 17 10:08:33.480153 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.480112    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b" containerName="revoke-keys"
Jul 17 10:08:33.484871 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.484849    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"
Jul 17 10:08:33.492113 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.492042    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"]
Jul 17 10:08:33.575989 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.575949    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b56e7e1-1388-4fa5-a652-9c1e46320c7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-28whn\" (UID: \"5b56e7e1-1388-4fa5-a652-9c1e46320c7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"
Jul 17 10:08:33.676889 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.676843    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b56e7e1-1388-4fa5-a652-9c1e46320c7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-28whn\" (UID: \"5b56e7e1-1388-4fa5-a652-9c1e46320c7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"
Jul 17 10:08:33.677515 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.677495    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b56e7e1-1388-4fa5-a652-9c1e46320c7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-28whn\" (UID: \"5b56e7e1-1388-4fa5-a652-9c1e46320c7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"
Jul 17 10:08:33.799191 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.799106    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"
Jul 17 10:08:33.939715 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:33.939681    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"]
Jul 17 10:08:33.941238 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:08:33.941213    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b56e7e1_1388_4fa5_a652_9c1e46320c7c.slice/crio-b706ee8e8958789198b2d0ccbac7f02bfa4867034494fcfcbb17b28f8189aea0 WatchSource:0}: Error finding container b706ee8e8958789198b2d0ccbac7f02bfa4867034494fcfcbb17b28f8189aea0: Status 404 returned error can't find the container with id b706ee8e8958789198b2d0ccbac7f02bfa4867034494fcfcbb17b28f8189aea0
Jul 17 10:08:34.183410 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:34.183379    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn" event={"ID":"5b56e7e1-1388-4fa5-a652-9c1e46320c7c","Type":"ContainerStarted","Data":"b706ee8e8958789198b2d0ccbac7f02bfa4867034494fcfcbb17b28f8189aea0"}
Jul 17 10:08:34.300574 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:34.300548    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"
Jul 17 10:08:34.382227 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:34.382188    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/266493d2-e072-4f39-9040-e72c23950fc1-maas-api-service-ca\") pod \"266493d2-e072-4f39-9040-e72c23950fc1\" (UID: \"266493d2-e072-4f39-9040-e72c23950fc1\") "
Jul 17 10:08:34.382578 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:34.382558    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/266493d2-e072-4f39-9040-e72c23950fc1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "266493d2-e072-4f39-9040-e72c23950fc1" (UID: "266493d2-e072-4f39-9040-e72c23950fc1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:34.483087 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:34.483057    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/266493d2-e072-4f39-9040-e72c23950fc1-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:08:35.190899 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:35.190876    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"
Jul 17 10:08:35.191287 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:35.190877    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn" event={"ID":"266493d2-e072-4f39-9040-e72c23950fc1","Type":"ContainerDied","Data":"c1085a3fea326154f2b1ded907cdd08fc1d80c9b3e947b5d745d373592aea7a9"}
Jul 17 10:08:35.191287 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:35.191009    2571 scope.go:117] "RemoveContainer" containerID="74e9b6b2d47362cdb270370b7dc66140ff0d08c079368aaca8a336728a378c16"
Jul 17 10:08:35.192587 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:35.192559    2571 generic.go:356] "Generic (PLEG): container finished" podID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerID="a4bde5cbe6396c9956d5236924870ef411f9ce37a17e7b33442f3684b76e4415" exitCode=6
Jul 17 10:08:35.192695 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:35.192626    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn" event={"ID":"5b56e7e1-1388-4fa5-a652-9c1e46320c7c","Type":"ContainerDied","Data":"a4bde5cbe6396c9956d5236924870ef411f9ce37a17e7b33442f3684b76e4415"}
Jul 17 10:08:35.192795 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:35.192781    2571 scope.go:117] "RemoveContainer" containerID="a4bde5cbe6396c9956d5236924870ef411f9ce37a17e7b33442f3684b76e4415"
Jul 17 10:08:35.226937 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:35.226910    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"]
Jul 17 10:08:35.232382 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:35.232358    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-55vxn"]
Jul 17 10:08:35.758727 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:35.758692    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="266493d2-e072-4f39-9040-e72c23950fc1" path="/var/lib/kubelet/pods/266493d2-e072-4f39-9040-e72c23950fc1/volumes"
Jul 17 10:08:36.201288 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:36.201256    2571 generic.go:356] "Generic (PLEG): container finished" podID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerID="3261bd1abf3ac583b5fb25840ca901dd04fcb455524fb6bb13cd900dd3bb7d91" exitCode=6
Jul 17 10:08:36.201774 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:36.201333    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn" event={"ID":"5b56e7e1-1388-4fa5-a652-9c1e46320c7c","Type":"ContainerDied","Data":"3261bd1abf3ac583b5fb25840ca901dd04fcb455524fb6bb13cd900dd3bb7d91"}
Jul 17 10:08:36.201774 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:36.201392    2571 scope.go:117] "RemoveContainer" containerID="a4bde5cbe6396c9956d5236924870ef411f9ce37a17e7b33442f3684b76e4415"
Jul 17 10:08:36.201774 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:36.201664    2571 scope.go:117] "RemoveContainer" containerID="3261bd1abf3ac583b5fb25840ca901dd04fcb455524fb6bb13cd900dd3bb7d91"
Jul 17 10:08:36.201942 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:36.201919    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-28whn_odh-ai-gateway-infra(5b56e7e1-1388-4fa5-a652-9c1e46320c7c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c"
Jul 17 10:08:37.209479 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:37.209437    2571 scope.go:117] "RemoveContainer" containerID="3261bd1abf3ac583b5fb25840ca901dd04fcb455524fb6bb13cd900dd3bb7d91"
Jul 17 10:08:37.209875 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:37.209643    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-28whn_odh-ai-gateway-infra(5b56e7e1-1388-4fa5-a652-9c1e46320c7c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c"
Jul 17 10:08:37.629005 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:37.628922    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 10:08:37.640938 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:37.640917    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 10:08:45.955021 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:45.954977    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"]
Jul 17 10:08:45.955437 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:45.955422    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="266493d2-e072-4f39-9040-e72c23950fc1" containerName="revoke-keys"
Jul 17 10:08:45.955503 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:45.955440    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="266493d2-e072-4f39-9040-e72c23950fc1" containerName="revoke-keys"
Jul 17 10:08:45.955503 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:45.955487    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="266493d2-e072-4f39-9040-e72c23950fc1" containerName="revoke-keys"
Jul 17 10:08:45.955503 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:45.955494    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="266493d2-e072-4f39-9040-e72c23950fc1" containerName="revoke-keys"
Jul 17 10:08:45.955595 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:45.955554    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="266493d2-e072-4f39-9040-e72c23950fc1" containerName="revoke-keys"
Jul 17 10:08:45.955595 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:45.955567    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="266493d2-e072-4f39-9040-e72c23950fc1" containerName="revoke-keys"
Jul 17 10:08:45.958638 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:45.958623    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"
Jul 17 10:08:45.968336 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:45.968311    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"]
Jul 17 10:08:46.091012 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:46.090981    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/169222e4-7a68-4da8-b848-2d0e6d2851d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh\" (UID: \"169222e4-7a68-4da8-b848-2d0e6d2851d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"
Jul 17 10:08:46.191860 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:46.191796    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/169222e4-7a68-4da8-b848-2d0e6d2851d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh\" (UID: \"169222e4-7a68-4da8-b848-2d0e6d2851d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"
Jul 17 10:08:46.192491 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:46.192442    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/169222e4-7a68-4da8-b848-2d0e6d2851d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh\" (UID: \"169222e4-7a68-4da8-b848-2d0e6d2851d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"
Jul 17 10:08:46.270233 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:46.270158    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"
Jul 17 10:08:46.416932 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:08:46.416900    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod169222e4_7a68_4da8_b848_2d0e6d2851d3.slice/crio-93fe545ab2b11280a0f002bb49f465ea5c52c4071270b674bafe3946b25ee427 WatchSource:0}: Error finding container 93fe545ab2b11280a0f002bb49f465ea5c52c4071270b674bafe3946b25ee427: Status 404 returned error can't find the container with id 93fe545ab2b11280a0f002bb49f465ea5c52c4071270b674bafe3946b25ee427
Jul 17 10:08:46.417091 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:46.417065    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"]
Jul 17 10:08:47.275944 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:47.275904    2571 generic.go:356] "Generic (PLEG): container finished" podID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerID="f30fa14755c3bf7e8fdd3dc89645490f550057b3fa829d82894206376d7739ff" exitCode=6
Jul 17 10:08:47.276314 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:47.275990    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh" event={"ID":"169222e4-7a68-4da8-b848-2d0e6d2851d3","Type":"ContainerDied","Data":"f30fa14755c3bf7e8fdd3dc89645490f550057b3fa829d82894206376d7739ff"}
Jul 17 10:08:47.276314 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:47.276079    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh" event={"ID":"169222e4-7a68-4da8-b848-2d0e6d2851d3","Type":"ContainerStarted","Data":"93fe545ab2b11280a0f002bb49f465ea5c52c4071270b674bafe3946b25ee427"}
Jul 17 10:08:47.276314 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:47.276200    2571 scope.go:117] "RemoveContainer" containerID="f30fa14755c3bf7e8fdd3dc89645490f550057b3fa829d82894206376d7739ff"
Jul 17 10:08:48.283149 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:48.283108    2571 generic.go:356] "Generic (PLEG): container finished" podID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerID="47f7bfa3327816a159652111b4fd40b96e9e9c08585f7a95112e908d61730fae" exitCode=6
Jul 17 10:08:48.283711 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:48.283241    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh" event={"ID":"169222e4-7a68-4da8-b848-2d0e6d2851d3","Type":"ContainerDied","Data":"47f7bfa3327816a159652111b4fd40b96e9e9c08585f7a95112e908d61730fae"}
Jul 17 10:08:48.283711 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:48.283294    2571 scope.go:117] "RemoveContainer" containerID="f30fa14755c3bf7e8fdd3dc89645490f550057b3fa829d82894206376d7739ff"
Jul 17 10:08:48.283711 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:48.283575    2571 scope.go:117] "RemoveContainer" containerID="47f7bfa3327816a159652111b4fd40b96e9e9c08585f7a95112e908d61730fae"
Jul 17 10:08:48.283897 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:48.283844    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh_odh-ai-gateway-infra(169222e4-7a68-4da8-b848-2d0e6d2851d3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3"
Jul 17 10:08:49.290874 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:49.290839    2571 scope.go:117] "RemoveContainer" containerID="47f7bfa3327816a159652111b4fd40b96e9e9c08585f7a95112e908d61730fae"
Jul 17 10:08:49.291231 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:49.291074    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh_odh-ai-gateway-infra(169222e4-7a68-4da8-b848-2d0e6d2851d3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3"
Jul 17 10:08:52.753152 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:52.753115    2571 scope.go:117] "RemoveContainer" containerID="3261bd1abf3ac583b5fb25840ca901dd04fcb455524fb6bb13cd900dd3bb7d91"
Jul 17 10:08:53.316246 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:53.316212    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn" event={"ID":"5b56e7e1-1388-4fa5-a652-9c1e46320c7c","Type":"ContainerStarted","Data":"5e31cc0d8d3578e224aad95288e6ab61b676de3fcf7934ec4c0045390b566767"}
Jul 17 10:08:53.344645 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:53.344588    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn" podStartSLOduration=20.34456649 podStartE2EDuration="20.34456649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:08:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:08:53.344189729 +0000 UTC m=+3017.409010451" watchObservedRunningTime="2026-07-17 10:08:53.34456649 +0000 UTC m=+3017.409387212"
Jul 17 10:08:54.324237 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.324202    2571 generic.go:356] "Generic (PLEG): container finished" podID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerID="5e31cc0d8d3578e224aad95288e6ab61b676de3fcf7934ec4c0045390b566767" exitCode=6
Jul 17 10:08:54.324724 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.324279    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn" event={"ID":"5b56e7e1-1388-4fa5-a652-9c1e46320c7c","Type":"ContainerDied","Data":"5e31cc0d8d3578e224aad95288e6ab61b676de3fcf7934ec4c0045390b566767"}
Jul 17 10:08:54.324724 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.324358    2571 scope.go:117] "RemoveContainer" containerID="3261bd1abf3ac583b5fb25840ca901dd04fcb455524fb6bb13cd900dd3bb7d91"
Jul 17 10:08:54.324724 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.324693    2571 scope.go:117] "RemoveContainer" containerID="5e31cc0d8d3578e224aad95288e6ab61b676de3fcf7934ec4c0045390b566767"
Jul 17 10:08:54.324940 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:54.324922    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-28whn_odh-ai-gateway-infra(5b56e7e1-1388-4fa5-a652-9c1e46320c7c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c"
Jul 17 10:08:54.355074 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.355046    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"]
Jul 17 10:08:54.519711 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.519677    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"]
Jul 17 10:08:54.520143 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.520130    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="266493d2-e072-4f39-9040-e72c23950fc1" containerName="revoke-keys"
Jul 17 10:08:54.520192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.520146    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="266493d2-e072-4f39-9040-e72c23950fc1" containerName="revoke-keys"
Jul 17 10:08:54.520228 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.520218    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="266493d2-e072-4f39-9040-e72c23950fc1" containerName="revoke-keys"
Jul 17 10:08:54.524698 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.524678    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"
Jul 17 10:08:54.534899 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.534875    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"]
Jul 17 10:08:54.566184 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.566151    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa09c86f-cbf5-45ad-b2fd-2c05d54a512c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf\" (UID: \"aa09c86f-cbf5-45ad-b2fd-2c05d54a512c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"
Jul 17 10:08:54.666949 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.666852    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa09c86f-cbf5-45ad-b2fd-2c05d54a512c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf\" (UID: \"aa09c86f-cbf5-45ad-b2fd-2c05d54a512c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"
Jul 17 10:08:54.667545 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.667528    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa09c86f-cbf5-45ad-b2fd-2c05d54a512c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf\" (UID: \"aa09c86f-cbf5-45ad-b2fd-2c05d54a512c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"
Jul 17 10:08:54.836594 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.836557    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"
Jul 17 10:08:54.988701 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:54.988675    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"]
Jul 17 10:08:54.989407 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:08:54.989381    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa09c86f_cbf5_45ad_b2fd_2c05d54a512c.slice/crio-acd60d68a9966a70c464059b9668197d7a43ad048f0d53090f1c8e56be3aca70 WatchSource:0}: Error finding container acd60d68a9966a70c464059b9668197d7a43ad048f0d53090f1c8e56be3aca70: Status 404 returned error can't find the container with id acd60d68a9966a70c464059b9668197d7a43ad048f0d53090f1c8e56be3aca70
Jul 17 10:08:55.332533 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:55.332498    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf" event={"ID":"aa09c86f-cbf5-45ad-b2fd-2c05d54a512c","Type":"ContainerStarted","Data":"2f96cfa4967166695243eaaf5c2db2949e4ef8b32e89f4dc8d01cd04f68f3877"}
Jul 17 10:08:55.332985 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:55.332562    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf" event={"ID":"aa09c86f-cbf5-45ad-b2fd-2c05d54a512c","Type":"ContainerStarted","Data":"acd60d68a9966a70c464059b9668197d7a43ad048f0d53090f1c8e56be3aca70"}
Jul 17 10:08:55.351620 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:55.351567    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf" podStartSLOduration=1.351546699 podStartE2EDuration="1.351546699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:08:55.350511094 +0000 UTC m=+3019.415331817" watchObservedRunningTime="2026-07-17 10:08:55.351546699 +0000 UTC m=+3019.416367420"
Jul 17 10:08:55.472755 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:55.472731    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"
Jul 17 10:08:55.573687 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:55.573657    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b56e7e1-1388-4fa5-a652-9c1e46320c7c-maas-api-service-ca\") pod \"5b56e7e1-1388-4fa5-a652-9c1e46320c7c\" (UID: \"5b56e7e1-1388-4fa5-a652-9c1e46320c7c\") "
Jul 17 10:08:55.574005 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:55.573982    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b56e7e1-1388-4fa5-a652-9c1e46320c7c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5b56e7e1-1388-4fa5-a652-9c1e46320c7c" (UID: "5b56e7e1-1388-4fa5-a652-9c1e46320c7c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:55.675233 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:55.675196    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b56e7e1-1388-4fa5-a652-9c1e46320c7c-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:08:56.341548 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:56.341512    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerID="2f96cfa4967166695243eaaf5c2db2949e4ef8b32e89f4dc8d01cd04f68f3877" exitCode=6
Jul 17 10:08:56.341992 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:56.341604    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf" event={"ID":"aa09c86f-cbf5-45ad-b2fd-2c05d54a512c","Type":"ContainerDied","Data":"2f96cfa4967166695243eaaf5c2db2949e4ef8b32e89f4dc8d01cd04f68f3877"}
Jul 17 10:08:56.341992 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:56.341773    2571 scope.go:117] "RemoveContainer" containerID="2f96cfa4967166695243eaaf5c2db2949e4ef8b32e89f4dc8d01cd04f68f3877"
Jul 17 10:08:56.343153 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:56.343138    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"
Jul 17 10:08:56.343153 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:56.343143    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn" event={"ID":"5b56e7e1-1388-4fa5-a652-9c1e46320c7c","Type":"ContainerDied","Data":"b706ee8e8958789198b2d0ccbac7f02bfa4867034494fcfcbb17b28f8189aea0"}
Jul 17 10:08:56.343322 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:56.343183    2571 scope.go:117] "RemoveContainer" containerID="5e31cc0d8d3578e224aad95288e6ab61b676de3fcf7934ec4c0045390b566767"
Jul 17 10:08:56.377242 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:56.377210    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"]
Jul 17 10:08:56.379824 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:56.379798    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-28whn"]
Jul 17 10:08:57.351657 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:57.351570    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerID="9c5b2da24d6d9e948767e69d30c825c6b0ec8f1424838d85b03d0f698beb7618" exitCode=6
Jul 17 10:08:57.351657 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:57.351620    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf" event={"ID":"aa09c86f-cbf5-45ad-b2fd-2c05d54a512c","Type":"ContainerDied","Data":"9c5b2da24d6d9e948767e69d30c825c6b0ec8f1424838d85b03d0f698beb7618"}
Jul 17 10:08:57.352095 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:57.351702    2571 scope.go:117] "RemoveContainer" containerID="2f96cfa4967166695243eaaf5c2db2949e4ef8b32e89f4dc8d01cd04f68f3877"
Jul 17 10:08:57.352095 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:57.351910    2571 scope.go:117] "RemoveContainer" containerID="9c5b2da24d6d9e948767e69d30c825c6b0ec8f1424838d85b03d0f698beb7618"
Jul 17 10:08:57.352193 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:57.352168    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf_odh-ai-gateway-infra(aa09c86f-cbf5-45ad-b2fd-2c05d54a512c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c"
Jul 17 10:08:57.758450 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:57.758415    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" path="/var/lib/kubelet/pods/5b56e7e1-1388-4fa5-a652-9c1e46320c7c/volumes"
Jul 17 10:08:58.359055 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:08:58.359023    2571 scope.go:117] "RemoveContainer" containerID="9c5b2da24d6d9e948767e69d30c825c6b0ec8f1424838d85b03d0f698beb7618"
Jul 17 10:08:58.359520 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:08:58.359269    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf_odh-ai-gateway-infra(aa09c86f-cbf5-45ad-b2fd-2c05d54a512c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c"
Jul 17 10:09:01.752219 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:01.752185    2571 scope.go:117] "RemoveContainer" containerID="47f7bfa3327816a159652111b4fd40b96e9e9c08585f7a95112e908d61730fae"
Jul 17 10:09:02.384827 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:02.384740    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh" event={"ID":"169222e4-7a68-4da8-b848-2d0e6d2851d3","Type":"ContainerStarted","Data":"39e9c03a8d05863bbf4ef55bc4d58378d1584484c5c0665d85a8faa019198da7"}
Jul 17 10:09:02.413190 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:02.413139    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh" podStartSLOduration=17.413124866 podStartE2EDuration="17.413124866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:09:02.412515883 +0000 UTC m=+3026.477336599" watchObservedRunningTime="2026-07-17 10:09:02.413124866 +0000 UTC m=+3026.477945637"
Jul 17 10:09:03.393607 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:03.393568    2571 generic.go:356] "Generic (PLEG): container finished" podID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerID="39e9c03a8d05863bbf4ef55bc4d58378d1584484c5c0665d85a8faa019198da7" exitCode=6
Jul 17 10:09:03.393607 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:03.393609    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh" event={"ID":"169222e4-7a68-4da8-b848-2d0e6d2851d3","Type":"ContainerDied","Data":"39e9c03a8d05863bbf4ef55bc4d58378d1584484c5c0665d85a8faa019198da7"}
Jul 17 10:09:03.394244 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:03.393655    2571 scope.go:117] "RemoveContainer" containerID="47f7bfa3327816a159652111b4fd40b96e9e9c08585f7a95112e908d61730fae"
Jul 17 10:09:03.394244 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:03.393919    2571 scope.go:117] "RemoveContainer" containerID="39e9c03a8d05863bbf4ef55bc4d58378d1584484c5c0665d85a8faa019198da7"
Jul 17 10:09:03.394244 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:09:03.394166    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh_odh-ai-gateway-infra(169222e4-7a68-4da8-b848-2d0e6d2851d3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3"
Jul 17 10:09:03.426695 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:03.426663    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"]
Jul 17 10:09:04.550983 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:04.550956    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"
Jul 17 10:09:04.655641 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:04.655565    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/169222e4-7a68-4da8-b848-2d0e6d2851d3-maas-api-service-ca\") pod \"169222e4-7a68-4da8-b848-2d0e6d2851d3\" (UID: \"169222e4-7a68-4da8-b848-2d0e6d2851d3\") "
Jul 17 10:09:04.655927 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:04.655906    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/169222e4-7a68-4da8-b848-2d0e6d2851d3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "169222e4-7a68-4da8-b848-2d0e6d2851d3" (UID: "169222e4-7a68-4da8-b848-2d0e6d2851d3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:09:04.756257 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:04.756221    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/169222e4-7a68-4da8-b848-2d0e6d2851d3-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:09:05.410558 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:05.410528    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"
Jul 17 10:09:05.410734 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:05.410534    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh" event={"ID":"169222e4-7a68-4da8-b848-2d0e6d2851d3","Type":"ContainerDied","Data":"93fe545ab2b11280a0f002bb49f465ea5c52c4071270b674bafe3946b25ee427"}
Jul 17 10:09:05.410734 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:05.410659    2571 scope.go:117] "RemoveContainer" containerID="39e9c03a8d05863bbf4ef55bc4d58378d1584484c5c0665d85a8faa019198da7"
Jul 17 10:09:05.437268 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:05.437237    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"]
Jul 17 10:09:05.442695 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:05.442666    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-ldhvh"]
Jul 17 10:09:05.758763 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:05.758729    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3" path="/var/lib/kubelet/pods/169222e4-7a68-4da8-b848-2d0e6d2851d3/volumes"
Jul 17 10:09:12.752232 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:12.752200    2571 scope.go:117] "RemoveContainer" containerID="9c5b2da24d6d9e948767e69d30c825c6b0ec8f1424838d85b03d0f698beb7618"
Jul 17 10:09:13.459047 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:13.459016    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerID="7ce3ff493b799656c6c893c159a592572571e51315fcde8c956d38be00715450" exitCode=6
Jul 17 10:09:13.459190 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:13.459088    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf" event={"ID":"aa09c86f-cbf5-45ad-b2fd-2c05d54a512c","Type":"ContainerDied","Data":"7ce3ff493b799656c6c893c159a592572571e51315fcde8c956d38be00715450"}
Jul 17 10:09:13.459190 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:13.459153    2571 scope.go:117] "RemoveContainer" containerID="9c5b2da24d6d9e948767e69d30c825c6b0ec8f1424838d85b03d0f698beb7618"
Jul 17 10:09:13.459417 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:13.459399    2571 scope.go:117] "RemoveContainer" containerID="7ce3ff493b799656c6c893c159a592572571e51315fcde8c956d38be00715450"
Jul 17 10:09:13.459690 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:09:13.459666    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf_odh-ai-gateway-infra(aa09c86f-cbf5-45ad-b2fd-2c05d54a512c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c"
Jul 17 10:09:14.502778 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:14.502743    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"]
Jul 17 10:09:14.647508 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:14.647484    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"
Jul 17 10:09:14.742081 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:14.742042    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa09c86f-cbf5-45ad-b2fd-2c05d54a512c-maas-api-service-ca\") pod \"aa09c86f-cbf5-45ad-b2fd-2c05d54a512c\" (UID: \"aa09c86f-cbf5-45ad-b2fd-2c05d54a512c\") "
Jul 17 10:09:14.742398 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:14.742373    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa09c86f-cbf5-45ad-b2fd-2c05d54a512c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" (UID: "aa09c86f-cbf5-45ad-b2fd-2c05d54a512c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:09:14.844059 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:14.843915    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa09c86f-cbf5-45ad-b2fd-2c05d54a512c-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:09:15.481669 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:15.481640    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"
Jul 17 10:09:15.481669 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:15.481650    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf" event={"ID":"aa09c86f-cbf5-45ad-b2fd-2c05d54a512c","Type":"ContainerDied","Data":"acd60d68a9966a70c464059b9668197d7a43ad048f0d53090f1c8e56be3aca70"}
Jul 17 10:09:15.481923 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:15.481708    2571 scope.go:117] "RemoveContainer" containerID="7ce3ff493b799656c6c893c159a592572571e51315fcde8c956d38be00715450"
Jul 17 10:09:15.506399 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:15.506364    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"]
Jul 17 10:09:15.510328 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:15.510299    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-fw5tf"]
Jul 17 10:09:15.758957 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:15.758877    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" path="/var/lib/kubelet/pods/aa09c86f-cbf5-45ad-b2fd-2c05d54a512c/volumes"
Jul 17 10:09:33.531623 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.531544    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"]
Jul 17 10:09:33.532044 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532027    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerName="revoke-keys"
Jul 17 10:09:33.532102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532048    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerName="revoke-keys"
Jul 17 10:09:33.532102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532058    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerName="revoke-keys"
Jul 17 10:09:33.532102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532064    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerName="revoke-keys"
Jul 17 10:09:33.532102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532071    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerName="revoke-keys"
Jul 17 10:09:33.532102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532077    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerName="revoke-keys"
Jul 17 10:09:33.532102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532087    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerName="revoke-keys"
Jul 17 10:09:33.532102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532093    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerName="revoke-keys"
Jul 17 10:09:33.532102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532100    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerName="revoke-keys"
Jul 17 10:09:33.532102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532106    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532113    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532118    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532129    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532138    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532152    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532158    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532223    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532229    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532234    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532241    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532248    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532254    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="169222e4-7a68-4da8-b848-2d0e6d2851d3" containerName="revoke-keys"
Jul 17 10:09:33.532372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.532260    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerName="revoke-keys"
Jul 17 10:09:33.536009 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.535987    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"
Jul 17 10:09:33.538835 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.538816    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:09:33.547374 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.547349    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"]
Jul 17 10:09:33.617174 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.617135    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/955d1b3c-0d43-4912-9932-986c0f28da75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs\" (UID: \"955d1b3c-0d43-4912-9932-986c0f28da75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"
Jul 17 10:09:33.717903 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.717852    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/955d1b3c-0d43-4912-9932-986c0f28da75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs\" (UID: \"955d1b3c-0d43-4912-9932-986c0f28da75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"
Jul 17 10:09:33.718553 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.718532    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/955d1b3c-0d43-4912-9932-986c0f28da75-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs\" (UID: \"955d1b3c-0d43-4912-9932-986c0f28da75\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"
Jul 17 10:09:33.848957 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.848862    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"
Jul 17 10:09:33.996756 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:33.996731    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"]
Jul 17 10:09:33.997888 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:09:33.997857    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955d1b3c_0d43_4912_9932_986c0f28da75.slice/crio-1a47637478eb21c3043ce34d9a6e28a99a11b7c47537a7d1d62f241635f5f613 WatchSource:0}: Error finding container 1a47637478eb21c3043ce34d9a6e28a99a11b7c47537a7d1d62f241635f5f613: Status 404 returned error can't find the container with id 1a47637478eb21c3043ce34d9a6e28a99a11b7c47537a7d1d62f241635f5f613
Jul 17 10:09:34.601016 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:34.600919    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs" event={"ID":"955d1b3c-0d43-4912-9932-986c0f28da75","Type":"ContainerStarted","Data":"036ca24ce641580265e794fc9bca702d27e5d19af760b8e8a82e36e981183c7a"}
Jul 17 10:09:34.601016 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:34.600989    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs" event={"ID":"955d1b3c-0d43-4912-9932-986c0f28da75","Type":"ContainerStarted","Data":"1a47637478eb21c3043ce34d9a6e28a99a11b7c47537a7d1d62f241635f5f613"}
Jul 17 10:09:34.632684 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:34.632629    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs" podStartSLOduration=1.632613335 podStartE2EDuration="1.632613335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:09:34.631852743 +0000 UTC m=+3058.696673467" watchObservedRunningTime="2026-07-17 10:09:34.632613335 +0000 UTC m=+3058.697434057"
Jul 17 10:09:35.607973 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:35.607939    2571 generic.go:356] "Generic (PLEG): container finished" podID="955d1b3c-0d43-4912-9932-986c0f28da75" containerID="036ca24ce641580265e794fc9bca702d27e5d19af760b8e8a82e36e981183c7a" exitCode=6
Jul 17 10:09:35.608384 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:35.608025    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs" event={"ID":"955d1b3c-0d43-4912-9932-986c0f28da75","Type":"ContainerDied","Data":"036ca24ce641580265e794fc9bca702d27e5d19af760b8e8a82e36e981183c7a"}
Jul 17 10:09:35.608384 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:35.608192    2571 scope.go:117] "RemoveContainer" containerID="036ca24ce641580265e794fc9bca702d27e5d19af760b8e8a82e36e981183c7a"
Jul 17 10:09:36.000334 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.000291    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"]
Jul 17 10:09:36.000859 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.000835    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerName="revoke-keys"
Jul 17 10:09:36.000859 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.000858    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerName="revoke-keys"
Jul 17 10:09:36.001031 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.000991    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b56e7e1-1388-4fa5-a652-9c1e46320c7c" containerName="revoke-keys"
Jul 17 10:09:36.001031 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.001009    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa09c86f-cbf5-45ad-b2fd-2c05d54a512c" containerName="revoke-keys"
Jul 17 10:09:36.004330 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.004314    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"
Jul 17 10:09:36.013730 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.013692    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"]
Jul 17 10:09:36.140215 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.140172    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4\" (UID: \"6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"
Jul 17 10:09:36.241697 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.241660    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4\" (UID: \"6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"
Jul 17 10:09:36.242345 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.242325    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4\" (UID: \"6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"
Jul 17 10:09:36.316062 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.316011    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"
Jul 17 10:09:36.617187 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.617096    2571 generic.go:356] "Generic (PLEG): container finished" podID="955d1b3c-0d43-4912-9932-986c0f28da75" containerID="15ad37b9bbc6f0d08b5449bbe3669297988f6d3eec05553be256f5c0028fdc5c" exitCode=6
Jul 17 10:09:36.617187 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.617140    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs" event={"ID":"955d1b3c-0d43-4912-9932-986c0f28da75","Type":"ContainerDied","Data":"15ad37b9bbc6f0d08b5449bbe3669297988f6d3eec05553be256f5c0028fdc5c"}
Jul 17 10:09:36.617700 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.617207    2571 scope.go:117] "RemoveContainer" containerID="036ca24ce641580265e794fc9bca702d27e5d19af760b8e8a82e36e981183c7a"
Jul 17 10:09:36.617700 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.617411    2571 scope.go:117] "RemoveContainer" containerID="15ad37b9bbc6f0d08b5449bbe3669297988f6d3eec05553be256f5c0028fdc5c"
Jul 17 10:09:36.617700 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:09:36.617674    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs_odh-ai-gateway-infra(955d1b3c-0d43-4912-9932-986c0f28da75)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs" podUID="955d1b3c-0d43-4912-9932-986c0f28da75"
Jul 17 10:09:36.667716 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:36.667683    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"]
Jul 17 10:09:36.669184 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:09:36.669150    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a4c5173_fbc6_4dba_bdf8_1d94ef22c57e.slice/crio-ec90ab50aab052d16ed25135a227bb504c60682e13ec0b7961826a0309f6fc50 WatchSource:0}: Error finding container ec90ab50aab052d16ed25135a227bb504c60682e13ec0b7961826a0309f6fc50: Status 404 returned error can't find the container with id ec90ab50aab052d16ed25135a227bb504c60682e13ec0b7961826a0309f6fc50
Jul 17 10:09:37.625239 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:37.625142    2571 scope.go:117] "RemoveContainer" containerID="15ad37b9bbc6f0d08b5449bbe3669297988f6d3eec05553be256f5c0028fdc5c"
Jul 17 10:09:37.625678 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:09:37.625388    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs_odh-ai-gateway-infra(955d1b3c-0d43-4912-9932-986c0f28da75)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs" podUID="955d1b3c-0d43-4912-9932-986c0f28da75"
Jul 17 10:09:37.626542 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:37.626519    2571 generic.go:356] "Generic (PLEG): container finished" podID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerID="47518c3ba1e6cc2f3f84115931db8672a99cb0de2624fd3ca05a81f2e7bf40d4" exitCode=6
Jul 17 10:09:37.626635 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:37.626602    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4" event={"ID":"6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e","Type":"ContainerDied","Data":"47518c3ba1e6cc2f3f84115931db8672a99cb0de2624fd3ca05a81f2e7bf40d4"}
Jul 17 10:09:37.626697 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:37.626651    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4" event={"ID":"6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e","Type":"ContainerStarted","Data":"ec90ab50aab052d16ed25135a227bb504c60682e13ec0b7961826a0309f6fc50"}
Jul 17 10:09:37.626823 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:37.626807    2571 scope.go:117] "RemoveContainer" containerID="47518c3ba1e6cc2f3f84115931db8672a99cb0de2624fd3ca05a81f2e7bf40d4"
Jul 17 10:09:38.635612 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:38.635511    2571 generic.go:356] "Generic (PLEG): container finished" podID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerID="58190ea02856f5a54bff5970cb66733736734ab331fc706cedd6b369cdecfca8" exitCode=6
Jul 17 10:09:38.635612 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:38.635576    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4" event={"ID":"6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e","Type":"ContainerDied","Data":"58190ea02856f5a54bff5970cb66733736734ab331fc706cedd6b369cdecfca8"}
Jul 17 10:09:38.636154 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:38.635641    2571 scope.go:117] "RemoveContainer" containerID="47518c3ba1e6cc2f3f84115931db8672a99cb0de2624fd3ca05a81f2e7bf40d4"
Jul 17 10:09:38.636154 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:38.635858    2571 scope.go:117] "RemoveContainer" containerID="58190ea02856f5a54bff5970cb66733736734ab331fc706cedd6b369cdecfca8"
Jul 17 10:09:38.636154 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:09:38.636138    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4_odh-ai-gateway-infra(6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e"
Jul 17 10:09:39.643604 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:39.643576    2571 scope.go:117] "RemoveContainer" containerID="58190ea02856f5a54bff5970cb66733736734ab331fc706cedd6b369cdecfca8"
Jul 17 10:09:39.644082 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:09:39.643859    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4_odh-ai-gateway-infra(6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e"
Jul 17 10:09:52.752534 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:52.752457    2571 scope.go:117] "RemoveContainer" containerID="15ad37b9bbc6f0d08b5449bbe3669297988f6d3eec05553be256f5c0028fdc5c"
Jul 17 10:09:53.730247 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:53.730216    2571 generic.go:356] "Generic (PLEG): container finished" podID="955d1b3c-0d43-4912-9932-986c0f28da75" containerID="94c50482c6d9df149c46a91df2be60eaf3dfe15259b6acb9fadb88cfe14ad607" exitCode=6
Jul 17 10:09:53.730419 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:53.730290    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs" event={"ID":"955d1b3c-0d43-4912-9932-986c0f28da75","Type":"ContainerDied","Data":"94c50482c6d9df149c46a91df2be60eaf3dfe15259b6acb9fadb88cfe14ad607"}
Jul 17 10:09:53.730419 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:53.730354    2571 scope.go:117] "RemoveContainer" containerID="15ad37b9bbc6f0d08b5449bbe3669297988f6d3eec05553be256f5c0028fdc5c"
Jul 17 10:09:53.730764 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:53.730739    2571 scope.go:117] "RemoveContainer" containerID="94c50482c6d9df149c46a91df2be60eaf3dfe15259b6acb9fadb88cfe14ad607"
Jul 17 10:09:53.731049 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:09:53.731022    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs_odh-ai-gateway-infra(955d1b3c-0d43-4912-9932-986c0f28da75)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs" podUID="955d1b3c-0d43-4912-9932-986c0f28da75"
Jul 17 10:09:54.752310 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:54.752280    2571 scope.go:117] "RemoveContainer" containerID="58190ea02856f5a54bff5970cb66733736734ab331fc706cedd6b369cdecfca8"
Jul 17 10:09:54.772765 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:54.772736    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"]
Jul 17 10:09:54.918613 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:54.918580    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"
Jul 17 10:09:55.017634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.017547    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/955d1b3c-0d43-4912-9932-986c0f28da75-maas-api-service-ca\") pod \"955d1b3c-0d43-4912-9932-986c0f28da75\" (UID: \"955d1b3c-0d43-4912-9932-986c0f28da75\") "
Jul 17 10:09:55.017968 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.017945    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/955d1b3c-0d43-4912-9932-986c0f28da75-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "955d1b3c-0d43-4912-9932-986c0f28da75" (UID: "955d1b3c-0d43-4912-9932-986c0f28da75"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:09:55.118215 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.118180    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/955d1b3c-0d43-4912-9932-986c0f28da75-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:09:55.734549 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.734506    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"]
Jul 17 10:09:55.735027 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.735008    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955d1b3c-0d43-4912-9932-986c0f28da75" containerName="revoke-keys"
Jul 17 10:09:55.735122 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.735029    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="955d1b3c-0d43-4912-9932-986c0f28da75" containerName="revoke-keys"
Jul 17 10:09:55.735122 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.735042    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955d1b3c-0d43-4912-9932-986c0f28da75" containerName="revoke-keys"
Jul 17 10:09:55.735122 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.735050    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="955d1b3c-0d43-4912-9932-986c0f28da75" containerName="revoke-keys"
Jul 17 10:09:55.735281 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.735185    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="955d1b3c-0d43-4912-9932-986c0f28da75" containerName="revoke-keys"
Jul 17 10:09:55.735281 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.735201    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="955d1b3c-0d43-4912-9932-986c0f28da75" containerName="revoke-keys"
Jul 17 10:09:55.738440 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.738419    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"
Jul 17 10:09:55.746669 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.746642    2571 generic.go:356] "Generic (PLEG): container finished" podID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerID="3402513db20bd2d7222518376acca409edbaf0f4ebb414a07e0df99d3a25ca30" exitCode=6
Jul 17 10:09:55.746801 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.746721    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4" event={"ID":"6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e","Type":"ContainerDied","Data":"3402513db20bd2d7222518376acca409edbaf0f4ebb414a07e0df99d3a25ca30"}
Jul 17 10:09:55.746868 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.746796    2571 scope.go:117] "RemoveContainer" containerID="58190ea02856f5a54bff5970cb66733736734ab331fc706cedd6b369cdecfca8"
Jul 17 10:09:55.747181 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.747124    2571 scope.go:117] "RemoveContainer" containerID="3402513db20bd2d7222518376acca409edbaf0f4ebb414a07e0df99d3a25ca30"
Jul 17 10:09:55.747409 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:09:55.747389    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4_odh-ai-gateway-infra(6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e"
Jul 17 10:09:55.750161 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.749813    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs" event={"ID":"955d1b3c-0d43-4912-9932-986c0f28da75","Type":"ContainerDied","Data":"1a47637478eb21c3043ce34d9a6e28a99a11b7c47537a7d1d62f241635f5f613"}
Jul 17 10:09:55.750161 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.749902    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"
Jul 17 10:09:55.762262 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.762235    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"]
Jul 17 10:09:55.766948 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.766921    2571 scope.go:117] "RemoveContainer" containerID="94c50482c6d9df149c46a91df2be60eaf3dfe15259b6acb9fadb88cfe14ad607"
Jul 17 10:09:55.821078 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.820995    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"]
Jul 17 10:09:55.826607 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.826565    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7447989-2bc1-4363-9f80-47349ae368ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc\" (UID: \"d7447989-2bc1-4363-9f80-47349ae368ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"
Jul 17 10:09:55.829094 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.829073    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-xxcfs"]
Jul 17 10:09:55.928016 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.927980    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7447989-2bc1-4363-9f80-47349ae368ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc\" (UID: \"d7447989-2bc1-4363-9f80-47349ae368ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"
Jul 17 10:09:55.928848 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:55.928823    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7447989-2bc1-4363-9f80-47349ae368ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc\" (UID: \"d7447989-2bc1-4363-9f80-47349ae368ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"
Jul 17 10:09:56.050743 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:56.050654    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"
Jul 17 10:09:56.199863 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:56.199833    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"]
Jul 17 10:09:56.200877 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:09:56.200848    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7447989_2bc1_4363_9f80_47349ae368ad.slice/crio-f90da4a4d4048b9cb167f86ea17e5356eb87c31c60d6705cff691cd349707e80 WatchSource:0}: Error finding container f90da4a4d4048b9cb167f86ea17e5356eb87c31c60d6705cff691cd349707e80: Status 404 returned error can't find the container with id f90da4a4d4048b9cb167f86ea17e5356eb87c31c60d6705cff691cd349707e80
Jul 17 10:09:56.760294 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:56.760196    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc" event={"ID":"d7447989-2bc1-4363-9f80-47349ae368ad","Type":"ContainerStarted","Data":"f8542b3e6720345c996a15301247b8727edf5ed9f88d044c7f4beb9491bd7967"}
Jul 17 10:09:56.760294 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:56.760249    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc" event={"ID":"d7447989-2bc1-4363-9f80-47349ae368ad","Type":"ContainerStarted","Data":"f90da4a4d4048b9cb167f86ea17e5356eb87c31c60d6705cff691cd349707e80"}
Jul 17 10:09:56.794878 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:56.794823    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc" podStartSLOduration=1.7948098209999999 podStartE2EDuration="1.794809821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:09:56.79293538 +0000 UTC m=+3080.857756103" watchObservedRunningTime="2026-07-17 10:09:56.794809821 +0000 UTC m=+3080.859630576"
Jul 17 10:09:56.809638 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:56.809609    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"]
Jul 17 10:09:57.002932 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.002905    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"
Jul 17 10:09:57.038422 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.038348    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e-maas-api-service-ca\") pod \"6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e\" (UID: \"6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e\") "
Jul 17 10:09:57.038708 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.038685    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" (UID: "6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:09:57.139909 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.139875    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:09:57.758664 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.758632    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="955d1b3c-0d43-4912-9932-986c0f28da75" path="/var/lib/kubelet/pods/955d1b3c-0d43-4912-9932-986c0f28da75/volumes"
Jul 17 10:09:57.768917 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.768885    2571 generic.go:356] "Generic (PLEG): container finished" podID="d7447989-2bc1-4363-9f80-47349ae368ad" containerID="f8542b3e6720345c996a15301247b8727edf5ed9f88d044c7f4beb9491bd7967" exitCode=6
Jul 17 10:09:57.769029 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.768950    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc" event={"ID":"d7447989-2bc1-4363-9f80-47349ae368ad","Type":"ContainerDied","Data":"f8542b3e6720345c996a15301247b8727edf5ed9f88d044c7f4beb9491bd7967"}
Jul 17 10:09:57.769166 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.769151    2571 scope.go:117] "RemoveContainer" containerID="f8542b3e6720345c996a15301247b8727edf5ed9f88d044c7f4beb9491bd7967"
Jul 17 10:09:57.770313 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.770294    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"
Jul 17 10:09:57.770313 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.770301    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4" event={"ID":"6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e","Type":"ContainerDied","Data":"ec90ab50aab052d16ed25135a227bb504c60682e13ec0b7961826a0309f6fc50"}
Jul 17 10:09:57.770498 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.770351    2571 scope.go:117] "RemoveContainer" containerID="3402513db20bd2d7222518376acca409edbaf0f4ebb414a07e0df99d3a25ca30"
Jul 17 10:09:57.813822 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.813790    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"]
Jul 17 10:09:57.817333 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:57.817310    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-gvnl4"]
Jul 17 10:09:58.778128 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:58.778094    2571 generic.go:356] "Generic (PLEG): container finished" podID="d7447989-2bc1-4363-9f80-47349ae368ad" containerID="24f64da781b804e703edc3cc80ea4f576e139b9cd877597828495dca7113e074" exitCode=6
Jul 17 10:09:58.778336 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:58.778179    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc" event={"ID":"d7447989-2bc1-4363-9f80-47349ae368ad","Type":"ContainerDied","Data":"24f64da781b804e703edc3cc80ea4f576e139b9cd877597828495dca7113e074"}
Jul 17 10:09:58.778336 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:58.778236    2571 scope.go:117] "RemoveContainer" containerID="f8542b3e6720345c996a15301247b8727edf5ed9f88d044c7f4beb9491bd7967"
Jul 17 10:09:58.778494 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:58.778456    2571 scope.go:117] "RemoveContainer" containerID="24f64da781b804e703edc3cc80ea4f576e139b9cd877597828495dca7113e074"
Jul 17 10:09:58.778706 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:09:58.778679    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc_odh-ai-gateway-infra(d7447989-2bc1-4363-9f80-47349ae368ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc" podUID="d7447989-2bc1-4363-9f80-47349ae368ad"
Jul 17 10:09:59.763608 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:59.763573    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" path="/var/lib/kubelet/pods/6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e/volumes"
Jul 17 10:09:59.787677 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:09:59.787642    2571 scope.go:117] "RemoveContainer" containerID="24f64da781b804e703edc3cc80ea4f576e139b9cd877597828495dca7113e074"
Jul 17 10:09:59.787908 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:09:59.787886    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc_odh-ai-gateway-infra(d7447989-2bc1-4363-9f80-47349ae368ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc" podUID="d7447989-2bc1-4363-9f80-47349ae368ad"
Jul 17 10:10:13.752588 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:13.752546    2571 scope.go:117] "RemoveContainer" containerID="24f64da781b804e703edc3cc80ea4f576e139b9cd877597828495dca7113e074"
Jul 17 10:10:14.880412 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:14.880378    2571 generic.go:356] "Generic (PLEG): container finished" podID="d7447989-2bc1-4363-9f80-47349ae368ad" containerID="40f208412cc9e68dff17ddd4b2f242f875b679c6a1588c76871d39f458d72edd" exitCode=6
Jul 17 10:10:14.880852 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:14.880422    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc" event={"ID":"d7447989-2bc1-4363-9f80-47349ae368ad","Type":"ContainerDied","Data":"40f208412cc9e68dff17ddd4b2f242f875b679c6a1588c76871d39f458d72edd"}
Jul 17 10:10:14.880852 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:14.880485    2571 scope.go:117] "RemoveContainer" containerID="24f64da781b804e703edc3cc80ea4f576e139b9cd877597828495dca7113e074"
Jul 17 10:10:14.880852 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:14.880803    2571 scope.go:117] "RemoveContainer" containerID="40f208412cc9e68dff17ddd4b2f242f875b679c6a1588c76871d39f458d72edd"
Jul 17 10:10:14.881062 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:10:14.881037    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc_odh-ai-gateway-infra(d7447989-2bc1-4363-9f80-47349ae368ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc" podUID="d7447989-2bc1-4363-9f80-47349ae368ad"
Jul 17 10:10:15.912604 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:15.912499    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"]
Jul 17 10:10:16.050963 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:16.050938    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"
Jul 17 10:10:16.114281 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:16.114245    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7447989-2bc1-4363-9f80-47349ae368ad-maas-api-service-ca\") pod \"d7447989-2bc1-4363-9f80-47349ae368ad\" (UID: \"d7447989-2bc1-4363-9f80-47349ae368ad\") "
Jul 17 10:10:16.114649 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:16.114626    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7447989-2bc1-4363-9f80-47349ae368ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d7447989-2bc1-4363-9f80-47349ae368ad" (UID: "d7447989-2bc1-4363-9f80-47349ae368ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:10:16.215640 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:16.215612    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7447989-2bc1-4363-9f80-47349ae368ad-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:10:16.898011 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:16.897983    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"
Jul 17 10:10:16.898209 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:16.897985    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc" event={"ID":"d7447989-2bc1-4363-9f80-47349ae368ad","Type":"ContainerDied","Data":"f90da4a4d4048b9cb167f86ea17e5356eb87c31c60d6705cff691cd349707e80"}
Jul 17 10:10:16.898209 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:16.898121    2571 scope.go:117] "RemoveContainer" containerID="40f208412cc9e68dff17ddd4b2f242f875b679c6a1588c76871d39f458d72edd"
Jul 17 10:10:16.935210 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:16.935185    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"]
Jul 17 10:10:16.943062 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:16.943042    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-5wcpc"]
Jul 17 10:10:17.758155 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:17.758120    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7447989-2bc1-4363-9f80-47349ae368ad" path="/var/lib/kubelet/pods/d7447989-2bc1-4363-9f80-47349ae368ad/volumes"
Jul 17 10:10:33.590873 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.590830    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"]
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591259    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerName="revoke-keys"
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591270    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerName="revoke-keys"
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591279    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955d1b3c-0d43-4912-9932-986c0f28da75" containerName="revoke-keys"
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591285    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="955d1b3c-0d43-4912-9932-986c0f28da75" containerName="revoke-keys"
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591300    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerName="revoke-keys"
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591308    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerName="revoke-keys"
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591314    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerName="revoke-keys"
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591319    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerName="revoke-keys"
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591331    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7447989-2bc1-4363-9f80-47349ae368ad" containerName="revoke-keys"
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591336    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7447989-2bc1-4363-9f80-47349ae368ad" containerName="revoke-keys"
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591346    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7447989-2bc1-4363-9f80-47349ae368ad" containerName="revoke-keys"
Jul 17 10:10:33.591344 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591351    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7447989-2bc1-4363-9f80-47349ae368ad" containerName="revoke-keys"
Jul 17 10:10:33.591796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591358    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7447989-2bc1-4363-9f80-47349ae368ad" containerName="revoke-keys"
Jul 17 10:10:33.591796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591363    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7447989-2bc1-4363-9f80-47349ae368ad" containerName="revoke-keys"
Jul 17 10:10:33.591796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591418    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7447989-2bc1-4363-9f80-47349ae368ad" containerName="revoke-keys"
Jul 17 10:10:33.591796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591426    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerName="revoke-keys"
Jul 17 10:10:33.591796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591433    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7447989-2bc1-4363-9f80-47349ae368ad" containerName="revoke-keys"
Jul 17 10:10:33.591796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591441    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="955d1b3c-0d43-4912-9932-986c0f28da75" containerName="revoke-keys"
Jul 17 10:10:33.591796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.591448    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerName="revoke-keys"
Jul 17 10:10:33.595938 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.595919    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"
Jul 17 10:10:33.599664 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.599639    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:10:33.610599 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.610567    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"]
Jul 17 10:10:33.684902 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.684857    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52e845ac-3466-43d5-a86f-5938db7ecf8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz\" (UID: \"52e845ac-3466-43d5-a86f-5938db7ecf8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"
Jul 17 10:10:33.786062 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.786025    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52e845ac-3466-43d5-a86f-5938db7ecf8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz\" (UID: \"52e845ac-3466-43d5-a86f-5938db7ecf8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"
Jul 17 10:10:33.786696 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.786675    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52e845ac-3466-43d5-a86f-5938db7ecf8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz\" (UID: \"52e845ac-3466-43d5-a86f-5938db7ecf8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"
Jul 17 10:10:33.907118 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:33.907028    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"
Jul 17 10:10:34.052089 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:34.052053    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"]
Jul 17 10:10:34.053560 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:10:34.053532    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e845ac_3466_43d5_a86f_5938db7ecf8a.slice/crio-9d693cade35e58632f9331fc8f2fbfad802c77012f047cc132531f75f7644f51 WatchSource:0}: Error finding container 9d693cade35e58632f9331fc8f2fbfad802c77012f047cc132531f75f7644f51: Status 404 returned error can't find the container with id 9d693cade35e58632f9331fc8f2fbfad802c77012f047cc132531f75f7644f51
Jul 17 10:10:35.011156 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:35.011075    2571 generic.go:356] "Generic (PLEG): container finished" podID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerID="69d2046804dd8202ef89a97149a2ee0ee1da8dab17ff314d1c215cb017ae1bc1" exitCode=6
Jul 17 10:10:35.011156 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:35.011149    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz" event={"ID":"52e845ac-3466-43d5-a86f-5938db7ecf8a","Type":"ContainerDied","Data":"69d2046804dd8202ef89a97149a2ee0ee1da8dab17ff314d1c215cb017ae1bc1"}
Jul 17 10:10:35.011560 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:35.011189    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz" event={"ID":"52e845ac-3466-43d5-a86f-5938db7ecf8a","Type":"ContainerStarted","Data":"9d693cade35e58632f9331fc8f2fbfad802c77012f047cc132531f75f7644f51"}
Jul 17 10:10:35.011560 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:35.011371    2571 scope.go:117] "RemoveContainer" containerID="69d2046804dd8202ef89a97149a2ee0ee1da8dab17ff314d1c215cb017ae1bc1"
Jul 17 10:10:36.022238 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.022150    2571 generic.go:356] "Generic (PLEG): container finished" podID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerID="84e67c2c95279454a4acb28caa46e2e26378b77a2e75282269c1dfce3ba7de41" exitCode=6
Jul 17 10:10:36.022238 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.022222    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz" event={"ID":"52e845ac-3466-43d5-a86f-5938db7ecf8a","Type":"ContainerDied","Data":"84e67c2c95279454a4acb28caa46e2e26378b77a2e75282269c1dfce3ba7de41"}
Jul 17 10:10:36.022728 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.022268    2571 scope.go:117] "RemoveContainer" containerID="69d2046804dd8202ef89a97149a2ee0ee1da8dab17ff314d1c215cb017ae1bc1"
Jul 17 10:10:36.022728 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.022435    2571 scope.go:117] "RemoveContainer" containerID="84e67c2c95279454a4acb28caa46e2e26378b77a2e75282269c1dfce3ba7de41"
Jul 17 10:10:36.022728 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:10:36.022707    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz_odh-ai-gateway-infra(52e845ac-3466-43d5-a86f-5938db7ecf8a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a"
Jul 17 10:10:36.045615 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.045584    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"]
Jul 17 10:10:36.046103 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.046091    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7447989-2bc1-4363-9f80-47349ae368ad" containerName="revoke-keys"
Jul 17 10:10:36.046146 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.046108    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a4c5173-fbc6-4dba-bdf8-1d94ef22c57e" containerName="revoke-keys"
Jul 17 10:10:36.050163 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.050147    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"
Jul 17 10:10:36.058310 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.057791    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"]
Jul 17 10:10:36.210797 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.210763    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/138c9811-7f66-4eb6-b3a5-96f88299e6cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d\" (UID: \"138c9811-7f66-4eb6-b3a5-96f88299e6cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"
Jul 17 10:10:36.312196 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.312105    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/138c9811-7f66-4eb6-b3a5-96f88299e6cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d\" (UID: \"138c9811-7f66-4eb6-b3a5-96f88299e6cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"
Jul 17 10:10:36.312826 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.312808    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/138c9811-7f66-4eb6-b3a5-96f88299e6cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d\" (UID: \"138c9811-7f66-4eb6-b3a5-96f88299e6cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"
Jul 17 10:10:36.362877 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.362841    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"
Jul 17 10:10:36.713028 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:36.713001    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"]
Jul 17 10:10:36.714381 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:10:36.714343    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod138c9811_7f66_4eb6_b3a5_96f88299e6cc.slice/crio-111ecfdc214f9faf35318405c59ca2b573bb76c9d3bac813f475943afd93ef95 WatchSource:0}: Error finding container 111ecfdc214f9faf35318405c59ca2b573bb76c9d3bac813f475943afd93ef95: Status 404 returned error can't find the container with id 111ecfdc214f9faf35318405c59ca2b573bb76c9d3bac813f475943afd93ef95
Jul 17 10:10:37.030393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:37.030303    2571 scope.go:117] "RemoveContainer" containerID="84e67c2c95279454a4acb28caa46e2e26378b77a2e75282269c1dfce3ba7de41"
Jul 17 10:10:37.030863 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:10:37.030597    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz_odh-ai-gateway-infra(52e845ac-3466-43d5-a86f-5938db7ecf8a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a"
Jul 17 10:10:37.031423 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:37.031397    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d" event={"ID":"138c9811-7f66-4eb6-b3a5-96f88299e6cc","Type":"ContainerStarted","Data":"111ecfdc214f9faf35318405c59ca2b573bb76c9d3bac813f475943afd93ef95"}
Jul 17 10:10:38.039578 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:38.039536    2571 generic.go:356] "Generic (PLEG): container finished" podID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerID="d7c15ea82a915d7ab9ae6260001c8901d45e4bb9b3ca775f672fa105b90df9f8" exitCode=6
Jul 17 10:10:38.039990 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:38.039624    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d" event={"ID":"138c9811-7f66-4eb6-b3a5-96f88299e6cc","Type":"ContainerDied","Data":"d7c15ea82a915d7ab9ae6260001c8901d45e4bb9b3ca775f672fa105b90df9f8"}
Jul 17 10:10:38.039990 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:38.039825    2571 scope.go:117] "RemoveContainer" containerID="d7c15ea82a915d7ab9ae6260001c8901d45e4bb9b3ca775f672fa105b90df9f8"
Jul 17 10:10:39.048014 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:39.047912    2571 generic.go:356] "Generic (PLEG): container finished" podID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerID="822c5d2655b790214b832f4bd66b0c1285b3e66e86c149fd022c3d7e5a8b7348" exitCode=6
Jul 17 10:10:39.048451 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:39.048001    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d" event={"ID":"138c9811-7f66-4eb6-b3a5-96f88299e6cc","Type":"ContainerDied","Data":"822c5d2655b790214b832f4bd66b0c1285b3e66e86c149fd022c3d7e5a8b7348"}
Jul 17 10:10:39.048451 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:39.048075    2571 scope.go:117] "RemoveContainer" containerID="d7c15ea82a915d7ab9ae6260001c8901d45e4bb9b3ca775f672fa105b90df9f8"
Jul 17 10:10:39.048451 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:39.048233    2571 scope.go:117] "RemoveContainer" containerID="822c5d2655b790214b832f4bd66b0c1285b3e66e86c149fd022c3d7e5a8b7348"
Jul 17 10:10:39.048617 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:10:39.048499    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d_odh-ai-gateway-infra(138c9811-7f66-4eb6-b3a5-96f88299e6cc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc"
Jul 17 10:10:40.055754 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:40.055727    2571 scope.go:117] "RemoveContainer" containerID="822c5d2655b790214b832f4bd66b0c1285b3e66e86c149fd022c3d7e5a8b7348"
Jul 17 10:10:40.056229 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:10:40.055953    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d_odh-ai-gateway-infra(138c9811-7f66-4eb6-b3a5-96f88299e6cc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc"
Jul 17 10:10:44.645476 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:44.645428    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"]
Jul 17 10:10:44.650682 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:44.650659    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"
Jul 17 10:10:44.659605 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:44.659578    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"]
Jul 17 10:10:44.684965 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:44.684928    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6866dbd-1cbf-44ff-ba61-cacb302ad4c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz\" (UID: \"e6866dbd-1cbf-44ff-ba61-cacb302ad4c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"
Jul 17 10:10:44.786088 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:44.786055    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6866dbd-1cbf-44ff-ba61-cacb302ad4c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz\" (UID: \"e6866dbd-1cbf-44ff-ba61-cacb302ad4c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"
Jul 17 10:10:44.786756 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:44.786734    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6866dbd-1cbf-44ff-ba61-cacb302ad4c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz\" (UID: \"e6866dbd-1cbf-44ff-ba61-cacb302ad4c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"
Jul 17 10:10:44.964218 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:44.964166    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"
Jul 17 10:10:45.105083 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:45.105060    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"]
Jul 17 10:10:45.107436 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:10:45.107409    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6866dbd_1cbf_44ff_ba61_cacb302ad4c8.slice/crio-7dab28faa5ea5635aff02d8001542c69d6f4d9d61b8f0acbc3b9e7eb41f35ccc WatchSource:0}: Error finding container 7dab28faa5ea5635aff02d8001542c69d6f4d9d61b8f0acbc3b9e7eb41f35ccc: Status 404 returned error can't find the container with id 7dab28faa5ea5635aff02d8001542c69d6f4d9d61b8f0acbc3b9e7eb41f35ccc
Jul 17 10:10:46.104158 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:46.104118    2571 generic.go:356] "Generic (PLEG): container finished" podID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerID="b38f769368cd2c01d0da73b6f029a7a34cdf53543c5818ee599920ae5d4900a0" exitCode=6
Jul 17 10:10:46.104552 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:46.104195    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz" event={"ID":"e6866dbd-1cbf-44ff-ba61-cacb302ad4c8","Type":"ContainerDied","Data":"b38f769368cd2c01d0da73b6f029a7a34cdf53543c5818ee599920ae5d4900a0"}
Jul 17 10:10:46.104552 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:46.104247    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz" event={"ID":"e6866dbd-1cbf-44ff-ba61-cacb302ad4c8","Type":"ContainerStarted","Data":"7dab28faa5ea5635aff02d8001542c69d6f4d9d61b8f0acbc3b9e7eb41f35ccc"}
Jul 17 10:10:46.104552 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:46.104411    2571 scope.go:117] "RemoveContainer" containerID="b38f769368cd2c01d0da73b6f029a7a34cdf53543c5818ee599920ae5d4900a0"
Jul 17 10:10:47.113010 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:47.112926    2571 generic.go:356] "Generic (PLEG): container finished" podID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerID="6ac73b18b99ada37558017675be8022140b383ec20bd704484329d3ced16e4c6" exitCode=6
Jul 17 10:10:47.113010 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:47.112987    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz" event={"ID":"e6866dbd-1cbf-44ff-ba61-cacb302ad4c8","Type":"ContainerDied","Data":"6ac73b18b99ada37558017675be8022140b383ec20bd704484329d3ced16e4c6"}
Jul 17 10:10:47.113438 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:47.113034    2571 scope.go:117] "RemoveContainer" containerID="b38f769368cd2c01d0da73b6f029a7a34cdf53543c5818ee599920ae5d4900a0"
Jul 17 10:10:47.113438 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:47.113203    2571 scope.go:117] "RemoveContainer" containerID="6ac73b18b99ada37558017675be8022140b383ec20bd704484329d3ced16e4c6"
Jul 17 10:10:47.113537 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:10:47.113451    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz_odh-ai-gateway-infra(e6866dbd-1cbf-44ff-ba61-cacb302ad4c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8"
Jul 17 10:10:48.127101 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:48.127064    2571 scope.go:117] "RemoveContainer" containerID="6ac73b18b99ada37558017675be8022140b383ec20bd704484329d3ced16e4c6"
Jul 17 10:10:48.127525 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:10:48.127291    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz_odh-ai-gateway-infra(e6866dbd-1cbf-44ff-ba61-cacb302ad4c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8"
Jul 17 10:10:50.752950 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:50.752911    2571 scope.go:117] "RemoveContainer" containerID="84e67c2c95279454a4acb28caa46e2e26378b77a2e75282269c1dfce3ba7de41"
Jul 17 10:10:51.146743 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:51.146703    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz" event={"ID":"52e845ac-3466-43d5-a86f-5938db7ecf8a","Type":"ContainerStarted","Data":"5b63a10ac45beae0fb7ad738cc2f8df29d46450a7a525632fb2b0a4d9240ba00"}
Jul 17 10:10:51.172581 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:51.172525    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz" podStartSLOduration=18.172506817 podStartE2EDuration="18.172506817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:10:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:10:51.168365911 +0000 UTC m=+3135.233186633" watchObservedRunningTime="2026-07-17 10:10:51.172506817 +0000 UTC m=+3135.237327539"
Jul 17 10:10:52.154576 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:52.154542    2571 generic.go:356] "Generic (PLEG): container finished" podID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerID="5b63a10ac45beae0fb7ad738cc2f8df29d46450a7a525632fb2b0a4d9240ba00" exitCode=6
Jul 17 10:10:52.155003 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:52.154610    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz" event={"ID":"52e845ac-3466-43d5-a86f-5938db7ecf8a","Type":"ContainerDied","Data":"5b63a10ac45beae0fb7ad738cc2f8df29d46450a7a525632fb2b0a4d9240ba00"}
Jul 17 10:10:52.155003 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:52.154677    2571 scope.go:117] "RemoveContainer" containerID="84e67c2c95279454a4acb28caa46e2e26378b77a2e75282269c1dfce3ba7de41"
Jul 17 10:10:52.155003 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:52.154968    2571 scope.go:117] "RemoveContainer" containerID="5b63a10ac45beae0fb7ad738cc2f8df29d46450a7a525632fb2b0a4d9240ba00"
Jul 17 10:10:52.155191 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:10:52.155170    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz_odh-ai-gateway-infra(52e845ac-3466-43d5-a86f-5938db7ecf8a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a"
Jul 17 10:10:52.183751 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:52.183721    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"]
Jul 17 10:10:53.304855 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:53.304828    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"
Jul 17 10:10:53.361656 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:53.361621    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52e845ac-3466-43d5-a86f-5938db7ecf8a-maas-api-service-ca\") pod \"52e845ac-3466-43d5-a86f-5938db7ecf8a\" (UID: \"52e845ac-3466-43d5-a86f-5938db7ecf8a\") "
Jul 17 10:10:53.362018 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:53.361997    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52e845ac-3466-43d5-a86f-5938db7ecf8a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "52e845ac-3466-43d5-a86f-5938db7ecf8a" (UID: "52e845ac-3466-43d5-a86f-5938db7ecf8a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:10:53.463016 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:53.462982    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/52e845ac-3466-43d5-a86f-5938db7ecf8a-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:10:54.169417 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:54.169333    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"
Jul 17 10:10:54.169578 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:54.169330    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz" event={"ID":"52e845ac-3466-43d5-a86f-5938db7ecf8a","Type":"ContainerDied","Data":"9d693cade35e58632f9331fc8f2fbfad802c77012f047cc132531f75f7644f51"}
Jul 17 10:10:54.169578 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:54.169481    2571 scope.go:117] "RemoveContainer" containerID="5b63a10ac45beae0fb7ad738cc2f8df29d46450a7a525632fb2b0a4d9240ba00"
Jul 17 10:10:54.194248 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:54.194209    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"]
Jul 17 10:10:54.197742 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:54.197715    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kgbrz"]
Jul 17 10:10:54.752798 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:54.752769    2571 scope.go:117] "RemoveContainer" containerID="822c5d2655b790214b832f4bd66b0c1285b3e66e86c149fd022c3d7e5a8b7348"
Jul 17 10:10:55.177886 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.177843    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d" event={"ID":"138c9811-7f66-4eb6-b3a5-96f88299e6cc","Type":"ContainerStarted","Data":"f9b8c67e2e3ad6c312eec99dfca80d963fc003324ce3c839eb1b7e70bc8bb04b"}
Jul 17 10:10:55.198293 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.198242    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d" podStartSLOduration=19.198227446 podStartE2EDuration="19.198227446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:10:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:10:55.198107874 +0000 UTC m=+3139.262928597" watchObservedRunningTime="2026-07-17 10:10:55.198227446 +0000 UTC m=+3139.263048168"
Jul 17 10:10:55.759733 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.759691    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a" path="/var/lib/kubelet/pods/52e845ac-3466-43d5-a86f-5938db7ecf8a/volumes"
Jul 17 10:10:55.801255 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.801225    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"]
Jul 17 10:10:55.801751 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.801733    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerName="revoke-keys"
Jul 17 10:10:55.801852 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.801753    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerName="revoke-keys"
Jul 17 10:10:55.801852 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.801793    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerName="revoke-keys"
Jul 17 10:10:55.801852 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.801803    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerName="revoke-keys"
Jul 17 10:10:55.802008 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.801913    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerName="revoke-keys"
Jul 17 10:10:55.802008 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.801927    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerName="revoke-keys"
Jul 17 10:10:55.805309 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.805288    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"
Jul 17 10:10:55.818873 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.818844    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"]
Jul 17 10:10:55.885711 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.885674    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c2fa03c-ca1f-43da-84e6-9cd01c1c6757-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9\" (UID: \"0c2fa03c-ca1f-43da-84e6-9cd01c1c6757\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"
Jul 17 10:10:55.986184 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.986153    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c2fa03c-ca1f-43da-84e6-9cd01c1c6757-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9\" (UID: \"0c2fa03c-ca1f-43da-84e6-9cd01c1c6757\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"
Jul 17 10:10:55.986833 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:55.986809    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c2fa03c-ca1f-43da-84e6-9cd01c1c6757-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9\" (UID: \"0c2fa03c-ca1f-43da-84e6-9cd01c1c6757\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"
Jul 17 10:10:56.115974 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:56.115856    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"
Jul 17 10:10:56.188971 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:56.188942    2571 generic.go:356] "Generic (PLEG): container finished" podID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerID="f9b8c67e2e3ad6c312eec99dfca80d963fc003324ce3c839eb1b7e70bc8bb04b" exitCode=6
Jul 17 10:10:56.189144 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:56.188990    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d" event={"ID":"138c9811-7f66-4eb6-b3a5-96f88299e6cc","Type":"ContainerDied","Data":"f9b8c67e2e3ad6c312eec99dfca80d963fc003324ce3c839eb1b7e70bc8bb04b"}
Jul 17 10:10:56.189144 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:56.189042    2571 scope.go:117] "RemoveContainer" containerID="822c5d2655b790214b832f4bd66b0c1285b3e66e86c149fd022c3d7e5a8b7348"
Jul 17 10:10:56.189337 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:56.189311    2571 scope.go:117] "RemoveContainer" containerID="f9b8c67e2e3ad6c312eec99dfca80d963fc003324ce3c839eb1b7e70bc8bb04b"
Jul 17 10:10:56.189632 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:10:56.189607    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d_odh-ai-gateway-infra(138c9811-7f66-4eb6-b3a5-96f88299e6cc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc"
Jul 17 10:10:56.216216 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:56.216175    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"]
Jul 17 10:10:56.271592 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:56.271562    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"]
Jul 17 10:10:56.272350 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:10:56.272323    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c2fa03c_ca1f_43da_84e6_9cd01c1c6757.slice/crio-3568cb1f26f07c7252e3524f11c3942097721c93f6b178292ac48f5b2747f41e WatchSource:0}: Error finding container 3568cb1f26f07c7252e3524f11c3942097721c93f6b178292ac48f5b2747f41e: Status 404 returned error can't find the container with id 3568cb1f26f07c7252e3524f11c3942097721c93f6b178292ac48f5b2747f41e
Jul 17 10:10:57.203181 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:57.203146    2571 generic.go:356] "Generic (PLEG): container finished" podID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerID="597954aa186d77175a025c9a70b090166347a6b4ce93a9824258f43323b707ce" exitCode=6
Jul 17 10:10:57.203487 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:57.203224    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9" event={"ID":"0c2fa03c-ca1f-43da-84e6-9cd01c1c6757","Type":"ContainerDied","Data":"597954aa186d77175a025c9a70b090166347a6b4ce93a9824258f43323b707ce"}
Jul 17 10:10:57.203487 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:57.203315    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9" event={"ID":"0c2fa03c-ca1f-43da-84e6-9cd01c1c6757","Type":"ContainerStarted","Data":"3568cb1f26f07c7252e3524f11c3942097721c93f6b178292ac48f5b2747f41e"}
Jul 17 10:10:57.203487 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:57.203377    2571 scope.go:117] "RemoveContainer" containerID="597954aa186d77175a025c9a70b090166347a6b4ce93a9824258f43323b707ce"
Jul 17 10:10:57.334400 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:57.334375    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"
Jul 17 10:10:57.402030 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:57.401996    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/138c9811-7f66-4eb6-b3a5-96f88299e6cc-maas-api-service-ca\") pod \"138c9811-7f66-4eb6-b3a5-96f88299e6cc\" (UID: \"138c9811-7f66-4eb6-b3a5-96f88299e6cc\") "
Jul 17 10:10:57.402360 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:57.402335    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/138c9811-7f66-4eb6-b3a5-96f88299e6cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "138c9811-7f66-4eb6-b3a5-96f88299e6cc" (UID: "138c9811-7f66-4eb6-b3a5-96f88299e6cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:10:57.502939 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:57.502911    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/138c9811-7f66-4eb6-b3a5-96f88299e6cc-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:10:58.210988 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:58.210954    2571 generic.go:356] "Generic (PLEG): container finished" podID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerID="de1acd350bea46207b4113de5b13e4720ae497711f58d5762860cfb2c6f2a72c" exitCode=6
Jul 17 10:10:58.211432 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:58.211042    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9" event={"ID":"0c2fa03c-ca1f-43da-84e6-9cd01c1c6757","Type":"ContainerDied","Data":"de1acd350bea46207b4113de5b13e4720ae497711f58d5762860cfb2c6f2a72c"}
Jul 17 10:10:58.211432 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:58.211098    2571 scope.go:117] "RemoveContainer" containerID="597954aa186d77175a025c9a70b090166347a6b4ce93a9824258f43323b707ce"
Jul 17 10:10:58.211432 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:58.211251    2571 scope.go:117] "RemoveContainer" containerID="de1acd350bea46207b4113de5b13e4720ae497711f58d5762860cfb2c6f2a72c"
Jul 17 10:10:58.211602 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:10:58.211500    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9_odh-ai-gateway-infra(0c2fa03c-ca1f-43da-84e6-9cd01c1c6757)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757"
Jul 17 10:10:58.212450 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:58.212435    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"
Jul 17 10:10:58.212450 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:58.212442    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d" event={"ID":"138c9811-7f66-4eb6-b3a5-96f88299e6cc","Type":"ContainerDied","Data":"111ecfdc214f9faf35318405c59ca2b573bb76c9d3bac813f475943afd93ef95"}
Jul 17 10:10:58.223056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:58.223039    2571 scope.go:117] "RemoveContainer" containerID="f9b8c67e2e3ad6c312eec99dfca80d963fc003324ce3c839eb1b7e70bc8bb04b"
Jul 17 10:10:58.253997 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:58.253964    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"]
Jul 17 10:10:58.260850 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:58.260817    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-hpb6d"]
Jul 17 10:10:59.220621 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:59.220595    2571 scope.go:117] "RemoveContainer" containerID="de1acd350bea46207b4113de5b13e4720ae497711f58d5762860cfb2c6f2a72c"
Jul 17 10:10:59.221102 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:10:59.220832    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9_odh-ai-gateway-infra(0c2fa03c-ca1f-43da-84e6-9cd01c1c6757)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757"
Jul 17 10:10:59.759170 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:10:59.759135    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" path="/var/lib/kubelet/pods/138c9811-7f66-4eb6-b3a5-96f88299e6cc/volumes"
Jul 17 10:11:01.752570 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:01.752541    2571 scope.go:117] "RemoveContainer" containerID="6ac73b18b99ada37558017675be8022140b383ec20bd704484329d3ced16e4c6"
Jul 17 10:11:02.248718 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:02.248674    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz" event={"ID":"e6866dbd-1cbf-44ff-ba61-cacb302ad4c8","Type":"ContainerStarted","Data":"653713e9a882d5b4f81ed30960ebfa7b1dcd35eb8072908fd8cb723f3fb09685"}
Jul 17 10:11:02.278331 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:02.278275    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz" podStartSLOduration=18.278260486 podStartE2EDuration="18.278260486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:10:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:11:02.276839056 +0000 UTC m=+3146.341659778" watchObservedRunningTime="2026-07-17 10:11:02.278260486 +0000 UTC m=+3146.343081270"
Jul 17 10:11:03.256591 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:03.256557    2571 generic.go:356] "Generic (PLEG): container finished" podID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerID="653713e9a882d5b4f81ed30960ebfa7b1dcd35eb8072908fd8cb723f3fb09685" exitCode=6
Jul 17 10:11:03.257027 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:03.256619    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz" event={"ID":"e6866dbd-1cbf-44ff-ba61-cacb302ad4c8","Type":"ContainerDied","Data":"653713e9a882d5b4f81ed30960ebfa7b1dcd35eb8072908fd8cb723f3fb09685"}
Jul 17 10:11:03.257027 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:03.256666    2571 scope.go:117] "RemoveContainer" containerID="6ac73b18b99ada37558017675be8022140b383ec20bd704484329d3ced16e4c6"
Jul 17 10:11:03.257027 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:03.256979    2571 scope.go:117] "RemoveContainer" containerID="653713e9a882d5b4f81ed30960ebfa7b1dcd35eb8072908fd8cb723f3fb09685"
Jul 17 10:11:03.257211 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:03.257186    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz_odh-ai-gateway-infra(e6866dbd-1cbf-44ff-ba61-cacb302ad4c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8"
Jul 17 10:11:03.306159 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:03.306113    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"]
Jul 17 10:11:04.412545 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:04.412519    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"
Jul 17 10:11:04.467561 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:04.467523    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6866dbd-1cbf-44ff-ba61-cacb302ad4c8-maas-api-service-ca\") pod \"e6866dbd-1cbf-44ff-ba61-cacb302ad4c8\" (UID: \"e6866dbd-1cbf-44ff-ba61-cacb302ad4c8\") "
Jul 17 10:11:04.467903 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:04.467878    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6866dbd-1cbf-44ff-ba61-cacb302ad4c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" (UID: "e6866dbd-1cbf-44ff-ba61-cacb302ad4c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:11:04.568674 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:04.568592    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6866dbd-1cbf-44ff-ba61-cacb302ad4c8-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:11:05.270904 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:05.270870    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"
Jul 17 10:11:05.270904 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:05.270898    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz" event={"ID":"e6866dbd-1cbf-44ff-ba61-cacb302ad4c8","Type":"ContainerDied","Data":"7dab28faa5ea5635aff02d8001542c69d6f4d9d61b8f0acbc3b9e7eb41f35ccc"}
Jul 17 10:11:05.271282 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:05.270957    2571 scope.go:117] "RemoveContainer" containerID="653713e9a882d5b4f81ed30960ebfa7b1dcd35eb8072908fd8cb723f3fb09685"
Jul 17 10:11:05.299058 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:05.299028    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"]
Jul 17 10:11:05.308993 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:05.308958    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-jb2hz"]
Jul 17 10:11:05.761422 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:05.761377    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" path="/var/lib/kubelet/pods/e6866dbd-1cbf-44ff-ba61-cacb302ad4c8/volumes"
Jul 17 10:11:07.260404 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260361    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gfmgs"]
Jul 17 10:11:07.260846 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260831    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerName="revoke-keys"
Jul 17 10:11:07.260900 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260851    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerName="revoke-keys"
Jul 17 10:11:07.260900 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260867    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerName="revoke-keys"
Jul 17 10:11:07.260900 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260873    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerName="revoke-keys"
Jul 17 10:11:07.260900 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260888    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerName="revoke-keys"
Jul 17 10:11:07.260900 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260895    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260904    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260910    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260919    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260927    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260939    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260944    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260954    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.260959    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.261025    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.261036    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.261042    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.261049    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="52e845ac-3466-43d5-a86f-5938db7ecf8a" containerName="revoke-keys"
Jul 17 10:11:07.261056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.261056    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerName="revoke-keys"
Jul 17 10:11:07.265528 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.265511    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs"
Jul 17 10:11:07.268401 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.268378    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-drv95\""
Jul 17 10:11:07.274867 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.274842    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gfmgs"]
Jul 17 10:11:07.395213 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.395174    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsrqp\" (UniqueName: \"kubernetes.io/projected/ec7f4422-c368-4d6b-af55-e439eef5d913-kube-api-access-qsrqp\") pod \"kuadrant-operator-catalog-gfmgs\" (UID: \"ec7f4422-c368-4d6b-af55-e439eef5d913\") " pod="kuadrant-system/kuadrant-operator-catalog-gfmgs"
Jul 17 10:11:07.496628 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.496593    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsrqp\" (UniqueName: \"kubernetes.io/projected/ec7f4422-c368-4d6b-af55-e439eef5d913-kube-api-access-qsrqp\") pod \"kuadrant-operator-catalog-gfmgs\" (UID: \"ec7f4422-c368-4d6b-af55-e439eef5d913\") " pod="kuadrant-system/kuadrant-operator-catalog-gfmgs"
Jul 17 10:11:07.508072 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.508046    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsrqp\" (UniqueName: \"kubernetes.io/projected/ec7f4422-c368-4d6b-af55-e439eef5d913-kube-api-access-qsrqp\") pod \"kuadrant-operator-catalog-gfmgs\" (UID: \"ec7f4422-c368-4d6b-af55-e439eef5d913\") " pod="kuadrant-system/kuadrant-operator-catalog-gfmgs"
Jul 17 10:11:07.577172 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.577078    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs"
Jul 17 10:11:07.731162 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.731126    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gfmgs"]
Jul 17 10:11:07.734573 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:11:07.734546    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec7f4422_c368_4d6b_af55_e439eef5d913.slice/crio-e743689fb53cd847bf20c40ce3ac460c9517cbad3beb3a3ef1bc868988ee8a66 WatchSource:0}: Error finding container e743689fb53cd847bf20c40ce3ac460c9517cbad3beb3a3ef1bc868988ee8a66: Status 404 returned error can't find the container with id e743689fb53cd847bf20c40ce3ac460c9517cbad3beb3a3ef1bc868988ee8a66
Jul 17 10:11:07.736347 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:07.736327    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 10:11:08.294921 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:08.294882    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs" event={"ID":"ec7f4422-c368-4d6b-af55-e439eef5d913","Type":"ContainerStarted","Data":"e743689fb53cd847bf20c40ce3ac460c9517cbad3beb3a3ef1bc868988ee8a66"}
Jul 17 10:11:10.309983 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:10.309944    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs" event={"ID":"ec7f4422-c368-4d6b-af55-e439eef5d913","Type":"ContainerStarted","Data":"aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875"}
Jul 17 10:11:10.373628 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:10.373570    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs" podStartSLOduration=1.240801041 podStartE2EDuration="3.37355354s" totalImagesPullingTime="2.132752499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:11:10.370644612 +0000 UTC m=+3154.435465335" watchObservedRunningTime="2026-07-17 10:11:10.37355354 +0000 UTC m=+3154.438374262"
Jul 17 10:11:10.753016 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:10.752982    2571 scope.go:117] "RemoveContainer" containerID="de1acd350bea46207b4113de5b13e4720ae497711f58d5762860cfb2c6f2a72c"
Jul 17 10:11:11.317939 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:11.317845    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9" event={"ID":"0c2fa03c-ca1f-43da-84e6-9cd01c1c6757","Type":"ContainerStarted","Data":"9167fb15cdf1af55db811dcad2a47d4b06d613cde927a9004771fbf0734acbf0"}
Jul 17 10:11:11.339729 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:11.339675    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9" podStartSLOduration=16.339658285 podStartE2EDuration="16.339658285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:10:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:11:11.336340849 +0000 UTC m=+3155.401161573" watchObservedRunningTime="2026-07-17 10:11:11.339658285 +0000 UTC m=+3155.404479007"
Jul 17 10:11:12.329788 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:12.329755    2571 generic.go:356] "Generic (PLEG): container finished" podID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerID="9167fb15cdf1af55db811dcad2a47d4b06d613cde927a9004771fbf0734acbf0" exitCode=6
Jul 17 10:11:12.330275 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:12.329798    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9" event={"ID":"0c2fa03c-ca1f-43da-84e6-9cd01c1c6757","Type":"ContainerDied","Data":"9167fb15cdf1af55db811dcad2a47d4b06d613cde927a9004771fbf0734acbf0"}
Jul 17 10:11:12.330275 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:12.329868    2571 scope.go:117] "RemoveContainer" containerID="de1acd350bea46207b4113de5b13e4720ae497711f58d5762860cfb2c6f2a72c"
Jul 17 10:11:12.330275 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:12.330159    2571 scope.go:117] "RemoveContainer" containerID="9167fb15cdf1af55db811dcad2a47d4b06d613cde927a9004771fbf0734acbf0"
Jul 17 10:11:12.330456 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:12.330430    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9_odh-ai-gateway-infra(0c2fa03c-ca1f-43da-84e6-9cd01c1c6757)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757"
Jul 17 10:11:12.353650 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:12.353626    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"]
Jul 17 10:11:13.481014 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:13.480984    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"
Jul 17 10:11:13.551226 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:13.551190    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c2fa03c-ca1f-43da-84e6-9cd01c1c6757-maas-api-service-ca\") pod \"0c2fa03c-ca1f-43da-84e6-9cd01c1c6757\" (UID: \"0c2fa03c-ca1f-43da-84e6-9cd01c1c6757\") "
Jul 17 10:11:13.551608 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:13.551588    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c2fa03c-ca1f-43da-84e6-9cd01c1c6757-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" (UID: "0c2fa03c-ca1f-43da-84e6-9cd01c1c6757"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:11:13.652488 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:13.652377    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c2fa03c-ca1f-43da-84e6-9cd01c1c6757-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:11:14.346392 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:14.346357    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"
Jul 17 10:11:14.346583 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:14.346395    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9" event={"ID":"0c2fa03c-ca1f-43da-84e6-9cd01c1c6757","Type":"ContainerDied","Data":"3568cb1f26f07c7252e3524f11c3942097721c93f6b178292ac48f5b2747f41e"}
Jul 17 10:11:14.346583 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:14.346454    2571 scope.go:117] "RemoveContainer" containerID="9167fb15cdf1af55db811dcad2a47d4b06d613cde927a9004771fbf0734acbf0"
Jul 17 10:11:14.368053 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:14.368023    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"]
Jul 17 10:11:14.370742 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:14.370716    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-n9gg9"]
Jul 17 10:11:15.759389 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:15.759358    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" path="/var/lib/kubelet/pods/0c2fa03c-ca1f-43da-84e6-9cd01c1c6757/volumes"
Jul 17 10:11:17.577384 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:17.577348    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs"
Jul 17 10:11:17.577824 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:17.577413    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs"
Jul 17 10:11:17.600589 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:17.600562    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs"
Jul 17 10:11:18.393641 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:18.393615    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs"
Jul 17 10:11:18.499892 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:18.499833    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gfmgs"]
Jul 17 10:11:20.381830 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:20.381789    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs" podUID="ec7f4422-c368-4d6b-af55-e439eef5d913" containerName="registry-server" containerID="cri-o://aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875" gracePeriod=2
Jul 17 10:11:20.625209 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:20.625184    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs"
Jul 17 10:11:20.719934 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:20.719902    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsrqp\" (UniqueName: \"kubernetes.io/projected/ec7f4422-c368-4d6b-af55-e439eef5d913-kube-api-access-qsrqp\") pod \"ec7f4422-c368-4d6b-af55-e439eef5d913\" (UID: \"ec7f4422-c368-4d6b-af55-e439eef5d913\") "
Jul 17 10:11:20.721949 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:20.721925    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec7f4422-c368-4d6b-af55-e439eef5d913-kube-api-access-qsrqp" (OuterVolumeSpecName: "kube-api-access-qsrqp") pod "ec7f4422-c368-4d6b-af55-e439eef5d913" (UID: "ec7f4422-c368-4d6b-af55-e439eef5d913"). InnerVolumeSpecName "kube-api-access-qsrqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:11:20.821633 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:20.821596    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsrqp\" (UniqueName: \"kubernetes.io/projected/ec7f4422-c368-4d6b-af55-e439eef5d913-kube-api-access-qsrqp\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:11:21.388544 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:21.388512    2571 generic.go:356] "Generic (PLEG): container finished" podID="ec7f4422-c368-4d6b-af55-e439eef5d913" containerID="aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875" exitCode=0
Jul 17 10:11:21.389046 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:21.388557    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs"
Jul 17 10:11:21.389046 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:21.388610    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs" event={"ID":"ec7f4422-c368-4d6b-af55-e439eef5d913","Type":"ContainerDied","Data":"aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875"}
Jul 17 10:11:21.389046 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:21.388659    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-gfmgs" event={"ID":"ec7f4422-c368-4d6b-af55-e439eef5d913","Type":"ContainerDied","Data":"e743689fb53cd847bf20c40ce3ac460c9517cbad3beb3a3ef1bc868988ee8a66"}
Jul 17 10:11:21.389046 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:21.388680    2571 scope.go:117] "RemoveContainer" containerID="aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875"
Jul 17 10:11:21.399056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:21.399038    2571 scope.go:117] "RemoveContainer" containerID="aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875"
Jul 17 10:11:21.399350 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:21.399332    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875\": container with ID starting with aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875 not found: ID does not exist" containerID="aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875"
Jul 17 10:11:21.399423 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:21.399360    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875"} err="failed to get container status \"aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875\": rpc error: code = NotFound desc = could not find container \"aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875\": container with ID starting with aa09c4ce9d5367c7c213d39b20ebd9f38423cbbb7693dfbe6ab41f861d0f4875 not found: ID does not exist"
Jul 17 10:11:21.419838 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:21.419790    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gfmgs"]
Jul 17 10:11:21.424169 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:21.424142    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-gfmgs"]
Jul 17 10:11:21.759167 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:21.759130    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec7f4422-c368-4d6b-af55-e439eef5d913" path="/var/lib/kubelet/pods/ec7f4422-c368-4d6b-af55-e439eef5d913/volumes"
Jul 17 10:11:33.691336 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691302    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"]
Jul 17 10:11:33.691780 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691766    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerName="revoke-keys"
Jul 17 10:11:33.691780 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691782    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerName="revoke-keys"
Jul 17 10:11:33.691881 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691811    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerName="revoke-keys"
Jul 17 10:11:33.691881 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691817    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerName="revoke-keys"
Jul 17 10:11:33.691881 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691828    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerName="revoke-keys"
Jul 17 10:11:33.691881 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691833    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerName="revoke-keys"
Jul 17 10:11:33.691881 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691843    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec7f4422-c368-4d6b-af55-e439eef5d913" containerName="registry-server"
Jul 17 10:11:33.691881 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691849    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec7f4422-c368-4d6b-af55-e439eef5d913" containerName="registry-server"
Jul 17 10:11:33.692060 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691917    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerName="revoke-keys"
Jul 17 10:11:33.692060 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691925    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec7f4422-c368-4d6b-af55-e439eef5d913" containerName="registry-server"
Jul 17 10:11:33.692060 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691932    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6866dbd-1cbf-44ff-ba61-cacb302ad4c8" containerName="revoke-keys"
Jul 17 10:11:33.692060 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691941    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="138c9811-7f66-4eb6-b3a5-96f88299e6cc" containerName="revoke-keys"
Jul 17 10:11:33.692060 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.691947    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerName="revoke-keys"
Jul 17 10:11:33.696396 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.696379    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"
Jul 17 10:11:33.700500 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.700480    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:11:33.712154 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.712125    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"]
Jul 17 10:11:33.739192 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.739157    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/147846c0-c5df-45f6-bce5-3c1ee7ffe019-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4\" (UID: \"147846c0-c5df-45f6-bce5-3c1ee7ffe019\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"
Jul 17 10:11:33.839979 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.839942    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/147846c0-c5df-45f6-bce5-3c1ee7ffe019-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4\" (UID: \"147846c0-c5df-45f6-bce5-3c1ee7ffe019\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"
Jul 17 10:11:33.840644 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:33.840623    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/147846c0-c5df-45f6-bce5-3c1ee7ffe019-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4\" (UID: \"147846c0-c5df-45f6-bce5-3c1ee7ffe019\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"
Jul 17 10:11:34.006037 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:34.005941    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"
Jul 17 10:11:34.147915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:34.147887    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"]
Jul 17 10:11:34.149570 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:11:34.149543    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod147846c0_c5df_45f6_bce5_3c1ee7ffe019.slice/crio-457b1410b9168c68dfcb5f42ab6a3e337dc890caa67e0059a32bd09648ee49ae WatchSource:0}: Error finding container 457b1410b9168c68dfcb5f42ab6a3e337dc890caa67e0059a32bd09648ee49ae: Status 404 returned error can't find the container with id 457b1410b9168c68dfcb5f42ab6a3e337dc890caa67e0059a32bd09648ee49ae
Jul 17 10:11:34.469025 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:34.468965    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4" event={"ID":"147846c0-c5df-45f6-bce5-3c1ee7ffe019","Type":"ContainerStarted","Data":"db2050406338def79666b80455aebcb59bc14f7c94202d4bd26057e518021e0c"}
Jul 17 10:11:34.469173 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:34.469044    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4" event={"ID":"147846c0-c5df-45f6-bce5-3c1ee7ffe019","Type":"ContainerStarted","Data":"457b1410b9168c68dfcb5f42ab6a3e337dc890caa67e0059a32bd09648ee49ae"}
Jul 17 10:11:34.490618 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:34.490565    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4" podStartSLOduration=1.490539018 podStartE2EDuration="1.490539018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:11:34.487676114 +0000 UTC m=+3178.552496859" watchObservedRunningTime="2026-07-17 10:11:34.490539018 +0000 UTC m=+3178.555359738"
Jul 17 10:11:35.477019 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:35.476983    2571 generic.go:356] "Generic (PLEG): container finished" podID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerID="db2050406338def79666b80455aebcb59bc14f7c94202d4bd26057e518021e0c" exitCode=6
Jul 17 10:11:35.477424 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:35.477077    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4" event={"ID":"147846c0-c5df-45f6-bce5-3c1ee7ffe019","Type":"ContainerDied","Data":"db2050406338def79666b80455aebcb59bc14f7c94202d4bd26057e518021e0c"}
Jul 17 10:11:35.477424 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:35.477252    2571 scope.go:117] "RemoveContainer" containerID="db2050406338def79666b80455aebcb59bc14f7c94202d4bd26057e518021e0c"
Jul 17 10:11:36.143037 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.143002    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"]
Jul 17 10:11:36.143682 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.143656    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c2fa03c-ca1f-43da-84e6-9cd01c1c6757" containerName="revoke-keys"
Jul 17 10:11:36.146802 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.146787    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"
Jul 17 10:11:36.158306 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.158285    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/852364b4-bdda-4adb-a253-9dc14bd02054-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc\" (UID: \"852364b4-bdda-4adb-a253-9dc14bd02054\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"
Jul 17 10:11:36.164726 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.164690    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"]
Jul 17 10:11:36.259323 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.259285    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/852364b4-bdda-4adb-a253-9dc14bd02054-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc\" (UID: \"852364b4-bdda-4adb-a253-9dc14bd02054\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"
Jul 17 10:11:36.260036 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.260016    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/852364b4-bdda-4adb-a253-9dc14bd02054-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc\" (UID: \"852364b4-bdda-4adb-a253-9dc14bd02054\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"
Jul 17 10:11:36.456505 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.456474    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"
Jul 17 10:11:36.486250 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.486216    2571 generic.go:356] "Generic (PLEG): container finished" podID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerID="644f7d161294f879965db7e8badf27d1e621a6510c59d94043d2f7433d1386c4" exitCode=6
Jul 17 10:11:36.486708 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.486282    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4" event={"ID":"147846c0-c5df-45f6-bce5-3c1ee7ffe019","Type":"ContainerDied","Data":"644f7d161294f879965db7e8badf27d1e621a6510c59d94043d2f7433d1386c4"}
Jul 17 10:11:36.486708 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.486326    2571 scope.go:117] "RemoveContainer" containerID="db2050406338def79666b80455aebcb59bc14f7c94202d4bd26057e518021e0c"
Jul 17 10:11:36.486708 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.486574    2571 scope.go:117] "RemoveContainer" containerID="644f7d161294f879965db7e8badf27d1e621a6510c59d94043d2f7433d1386c4"
Jul 17 10:11:36.486884 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:36.486824    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4_odh-ai-gateway-infra(147846c0-c5df-45f6-bce5-3c1ee7ffe019)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019"
Jul 17 10:11:36.634744 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:36.634712    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"]
Jul 17 10:11:36.635488 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:11:36.635443    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852364b4_bdda_4adb_a253_9dc14bd02054.slice/crio-b64809ff3d7119ba36ee15ed484435812a6a28066bb515c70b2b8ca0f4be6b15 WatchSource:0}: Error finding container b64809ff3d7119ba36ee15ed484435812a6a28066bb515c70b2b8ca0f4be6b15: Status 404 returned error can't find the container with id b64809ff3d7119ba36ee15ed484435812a6a28066bb515c70b2b8ca0f4be6b15
Jul 17 10:11:37.493908 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:37.493808    2571 generic.go:356] "Generic (PLEG): container finished" podID="852364b4-bdda-4adb-a253-9dc14bd02054" containerID="b8e126451ab8be21c6491b2e6c90ead8d61a3e2365d4241a2253408ca848254e" exitCode=6
Jul 17 10:11:37.494369 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:37.493898    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc" event={"ID":"852364b4-bdda-4adb-a253-9dc14bd02054","Type":"ContainerDied","Data":"b8e126451ab8be21c6491b2e6c90ead8d61a3e2365d4241a2253408ca848254e"}
Jul 17 10:11:37.494369 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:37.493999    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc" event={"ID":"852364b4-bdda-4adb-a253-9dc14bd02054","Type":"ContainerStarted","Data":"b64809ff3d7119ba36ee15ed484435812a6a28066bb515c70b2b8ca0f4be6b15"}
Jul 17 10:11:37.494369 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:37.494089    2571 scope.go:117] "RemoveContainer" containerID="b8e126451ab8be21c6491b2e6c90ead8d61a3e2365d4241a2253408ca848254e"
Jul 17 10:11:37.495935 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:37.495916    2571 scope.go:117] "RemoveContainer" containerID="644f7d161294f879965db7e8badf27d1e621a6510c59d94043d2f7433d1386c4"
Jul 17 10:11:37.496100 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:37.496083    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4_odh-ai-gateway-infra(147846c0-c5df-45f6-bce5-3c1ee7ffe019)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019"
Jul 17 10:11:38.503676 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:38.503639    2571 generic.go:356] "Generic (PLEG): container finished" podID="852364b4-bdda-4adb-a253-9dc14bd02054" containerID="04cfadd980b5668fe3be11b24519936d34d9472184d85488e01444853ab53668" exitCode=6
Jul 17 10:11:38.504199 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:38.503724    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc" event={"ID":"852364b4-bdda-4adb-a253-9dc14bd02054","Type":"ContainerDied","Data":"04cfadd980b5668fe3be11b24519936d34d9472184d85488e01444853ab53668"}
Jul 17 10:11:38.504199 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:38.503825    2571 scope.go:117] "RemoveContainer" containerID="b8e126451ab8be21c6491b2e6c90ead8d61a3e2365d4241a2253408ca848254e"
Jul 17 10:11:38.504199 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:38.503985    2571 scope.go:117] "RemoveContainer" containerID="04cfadd980b5668fe3be11b24519936d34d9472184d85488e01444853ab53668"
Jul 17 10:11:38.504332 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:38.504229    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc_odh-ai-gateway-infra(852364b4-bdda-4adb-a253-9dc14bd02054)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc" podUID="852364b4-bdda-4adb-a253-9dc14bd02054"
Jul 17 10:11:39.511689 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:39.511663    2571 scope.go:117] "RemoveContainer" containerID="04cfadd980b5668fe3be11b24519936d34d9472184d85488e01444853ab53668"
Jul 17 10:11:39.512115 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:39.511908    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc_odh-ai-gateway-infra(852364b4-bdda-4adb-a253-9dc14bd02054)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc" podUID="852364b4-bdda-4adb-a253-9dc14bd02054"
Jul 17 10:11:44.718536 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:44.718498    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"]
Jul 17 10:11:44.723670 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:44.723648    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"
Jul 17 10:11:44.731242 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:44.731129    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"]
Jul 17 10:11:44.840582 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:44.840546    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c35f0b-a90b-4cd5-8b67-db3d3f17627b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h\" (UID: \"70c35f0b-a90b-4cd5-8b67-db3d3f17627b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"
Jul 17 10:11:44.942086 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:44.942045    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c35f0b-a90b-4cd5-8b67-db3d3f17627b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h\" (UID: \"70c35f0b-a90b-4cd5-8b67-db3d3f17627b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"
Jul 17 10:11:44.942775 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:44.942748    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c35f0b-a90b-4cd5-8b67-db3d3f17627b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h\" (UID: \"70c35f0b-a90b-4cd5-8b67-db3d3f17627b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"
Jul 17 10:11:45.036017 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:45.035921    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"
Jul 17 10:11:45.186369 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:11:45.186338    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c35f0b_a90b_4cd5_8b67_db3d3f17627b.slice/crio-b2f5b8cb86ffe3166f47413d54d8c936ba00e0de0a2ff11cda08ea8022daea60 WatchSource:0}: Error finding container b2f5b8cb86ffe3166f47413d54d8c936ba00e0de0a2ff11cda08ea8022daea60: Status 404 returned error can't find the container with id b2f5b8cb86ffe3166f47413d54d8c936ba00e0de0a2ff11cda08ea8022daea60
Jul 17 10:11:45.187126 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:45.187097    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"]
Jul 17 10:11:45.549761 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:45.549719    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h" event={"ID":"70c35f0b-a90b-4cd5-8b67-db3d3f17627b","Type":"ContainerStarted","Data":"6aaa986943fa6fe77412d43f66e4b463b0c0991be976565097e95071d7c37132"}
Jul 17 10:11:45.549761 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:45.549769    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h" event={"ID":"70c35f0b-a90b-4cd5-8b67-db3d3f17627b","Type":"ContainerStarted","Data":"b2f5b8cb86ffe3166f47413d54d8c936ba00e0de0a2ff11cda08ea8022daea60"}
Jul 17 10:11:45.864482 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:45.864419    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h" podStartSLOduration=1.864399433 podStartE2EDuration="1.864399433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:11:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:11:45.575859819 +0000 UTC m=+3189.640680542" watchObservedRunningTime="2026-07-17 10:11:45.864399433 +0000 UTC m=+3189.929220155"
Jul 17 10:11:45.866808 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:45.866784    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"]
Jul 17 10:11:45.870913 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:45.870897    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"
Jul 17 10:11:45.889246 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:45.889217    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"]
Jul 17 10:11:45.950879 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:45.950843    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/391e2ee4-3aca-40d0-8b53-73afe862e7c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm\" (UID: \"391e2ee4-3aca-40d0-8b53-73afe862e7c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"
Jul 17 10:11:46.051914 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:46.051818    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/391e2ee4-3aca-40d0-8b53-73afe862e7c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm\" (UID: \"391e2ee4-3aca-40d0-8b53-73afe862e7c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"
Jul 17 10:11:46.052555 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:46.052534    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/391e2ee4-3aca-40d0-8b53-73afe862e7c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm\" (UID: \"391e2ee4-3aca-40d0-8b53-73afe862e7c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"
Jul 17 10:11:46.181614 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:46.181581    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"
Jul 17 10:11:46.327239 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:46.327160    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"]
Jul 17 10:11:46.330320 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:11:46.330291    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod391e2ee4_3aca_40d0_8b53_73afe862e7c5.slice/crio-a067be9c6acc43d7c39cbc0f88339156334beab5df50f74fc1f17ab3a563f1fe WatchSource:0}: Error finding container a067be9c6acc43d7c39cbc0f88339156334beab5df50f74fc1f17ab3a563f1fe: Status 404 returned error can't find the container with id a067be9c6acc43d7c39cbc0f88339156334beab5df50f74fc1f17ab3a563f1fe
Jul 17 10:11:46.556774 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:46.556746    2571 generic.go:356] "Generic (PLEG): container finished" podID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerID="6aaa986943fa6fe77412d43f66e4b463b0c0991be976565097e95071d7c37132" exitCode=6
Jul 17 10:11:46.556895 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:46.556822    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h" event={"ID":"70c35f0b-a90b-4cd5-8b67-db3d3f17627b","Type":"ContainerDied","Data":"6aaa986943fa6fe77412d43f66e4b463b0c0991be976565097e95071d7c37132"}
Jul 17 10:11:46.557062 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:46.557042    2571 scope.go:117] "RemoveContainer" containerID="6aaa986943fa6fe77412d43f66e4b463b0c0991be976565097e95071d7c37132"
Jul 17 10:11:46.558282 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:46.558264    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm" event={"ID":"391e2ee4-3aca-40d0-8b53-73afe862e7c5","Type":"ContainerStarted","Data":"a067be9c6acc43d7c39cbc0f88339156334beab5df50f74fc1f17ab3a563f1fe"}
Jul 17 10:11:47.567196 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:47.567110    2571 generic.go:356] "Generic (PLEG): container finished" podID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerID="7695bd5bde101fdb956e8971535cafb41e3a087b81aa164dd2063150ce2cce28" exitCode=6
Jul 17 10:11:47.567618 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:47.567198    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm" event={"ID":"391e2ee4-3aca-40d0-8b53-73afe862e7c5","Type":"ContainerDied","Data":"7695bd5bde101fdb956e8971535cafb41e3a087b81aa164dd2063150ce2cce28"}
Jul 17 10:11:47.567618 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:47.567362    2571 scope.go:117] "RemoveContainer" containerID="7695bd5bde101fdb956e8971535cafb41e3a087b81aa164dd2063150ce2cce28"
Jul 17 10:11:47.569246 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:47.569223    2571 generic.go:356] "Generic (PLEG): container finished" podID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerID="662d1f700b82f97c5e5654d8b6db119e88fb861795b972d3eaaca67075c98b9d" exitCode=6
Jul 17 10:11:47.569329 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:47.569284    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h" event={"ID":"70c35f0b-a90b-4cd5-8b67-db3d3f17627b","Type":"ContainerDied","Data":"662d1f700b82f97c5e5654d8b6db119e88fb861795b972d3eaaca67075c98b9d"}
Jul 17 10:11:47.569368 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:47.569330    2571 scope.go:117] "RemoveContainer" containerID="6aaa986943fa6fe77412d43f66e4b463b0c0991be976565097e95071d7c37132"
Jul 17 10:11:47.569527 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:47.569510    2571 scope.go:117] "RemoveContainer" containerID="662d1f700b82f97c5e5654d8b6db119e88fb861795b972d3eaaca67075c98b9d"
Jul 17 10:11:47.569770 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:47.569740    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h_odh-ai-gateway-infra(70c35f0b-a90b-4cd5-8b67-db3d3f17627b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b"
Jul 17 10:11:48.577185 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:48.577152    2571 scope.go:117] "RemoveContainer" containerID="662d1f700b82f97c5e5654d8b6db119e88fb861795b972d3eaaca67075c98b9d"
Jul 17 10:11:48.577633 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:48.577379    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h_odh-ai-gateway-infra(70c35f0b-a90b-4cd5-8b67-db3d3f17627b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b"
Jul 17 10:11:48.578650 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:48.578628    2571 generic.go:356] "Generic (PLEG): container finished" podID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerID="49933a126b03b93b55d71fe3a64a6f2959a2e1c9df0f1c8e2e103c16fd076779" exitCode=6
Jul 17 10:11:48.578759 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:48.578710    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm" event={"ID":"391e2ee4-3aca-40d0-8b53-73afe862e7c5","Type":"ContainerDied","Data":"49933a126b03b93b55d71fe3a64a6f2959a2e1c9df0f1c8e2e103c16fd076779"}
Jul 17 10:11:48.578809 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:48.578784    2571 scope.go:117] "RemoveContainer" containerID="7695bd5bde101fdb956e8971535cafb41e3a087b81aa164dd2063150ce2cce28"
Jul 17 10:11:48.578866 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:48.578849    2571 scope.go:117] "RemoveContainer" containerID="49933a126b03b93b55d71fe3a64a6f2959a2e1c9df0f1c8e2e103c16fd076779"
Jul 17 10:11:48.579014 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:48.578993    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm_odh-ai-gateway-infra(391e2ee4-3aca-40d0-8b53-73afe862e7c5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5"
Jul 17 10:11:49.586636 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:49.586603    2571 scope.go:117] "RemoveContainer" containerID="49933a126b03b93b55d71fe3a64a6f2959a2e1c9df0f1c8e2e103c16fd076779"
Jul 17 10:11:49.587019 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:49.586776    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm_odh-ai-gateway-infra(391e2ee4-3aca-40d0-8b53-73afe862e7c5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5"
Jul 17 10:11:50.752098 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:50.752067    2571 scope.go:117] "RemoveContainer" containerID="644f7d161294f879965db7e8badf27d1e621a6510c59d94043d2f7433d1386c4"
Jul 17 10:11:51.599558 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:51.599445    2571 generic.go:356] "Generic (PLEG): container finished" podID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerID="c18c7c558b091ae104032c781675e684f22df6c99e8b94dc99870909ccab89a1" exitCode=6
Jul 17 10:11:51.599558 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:51.599523    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4" event={"ID":"147846c0-c5df-45f6-bce5-3c1ee7ffe019","Type":"ContainerDied","Data":"c18c7c558b091ae104032c781675e684f22df6c99e8b94dc99870909ccab89a1"}
Jul 17 10:11:51.599818 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:51.599625    2571 scope.go:117] "RemoveContainer" containerID="644f7d161294f879965db7e8badf27d1e621a6510c59d94043d2f7433d1386c4"
Jul 17 10:11:51.600011 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:51.599994    2571 scope.go:117] "RemoveContainer" containerID="c18c7c558b091ae104032c781675e684f22df6c99e8b94dc99870909ccab89a1"
Jul 17 10:11:51.600236 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:51.600217    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4_odh-ai-gateway-infra(147846c0-c5df-45f6-bce5-3c1ee7ffe019)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019"
Jul 17 10:11:51.752621 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:51.752572    2571 scope.go:117] "RemoveContainer" containerID="04cfadd980b5668fe3be11b24519936d34d9472184d85488e01444853ab53668"
Jul 17 10:11:52.609059 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:52.608985    2571 generic.go:356] "Generic (PLEG): container finished" podID="852364b4-bdda-4adb-a253-9dc14bd02054" containerID="6836a84c6d1c7076fd908ba7c3375b8e8054f15c571225467f73b910a05e304b" exitCode=6
Jul 17 10:11:52.609059 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:52.609042    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc" event={"ID":"852364b4-bdda-4adb-a253-9dc14bd02054","Type":"ContainerDied","Data":"6836a84c6d1c7076fd908ba7c3375b8e8054f15c571225467f73b910a05e304b"}
Jul 17 10:11:52.609262 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:52.609086    2571 scope.go:117] "RemoveContainer" containerID="04cfadd980b5668fe3be11b24519936d34d9472184d85488e01444853ab53668"
Jul 17 10:11:52.609445 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:52.609420    2571 scope.go:117] "RemoveContainer" containerID="6836a84c6d1c7076fd908ba7c3375b8e8054f15c571225467f73b910a05e304b"
Jul 17 10:11:52.609694 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:11:52.609675    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc_odh-ai-gateway-infra(852364b4-bdda-4adb-a253-9dc14bd02054)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc" podUID="852364b4-bdda-4adb-a253-9dc14bd02054"
Jul 17 10:11:52.652333 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:52.652303    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"]
Jul 17 10:11:52.797886 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:52.797861    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"
Jul 17 10:11:52.916990 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:52.916896    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/147846c0-c5df-45f6-bce5-3c1ee7ffe019-maas-api-service-ca\") pod \"147846c0-c5df-45f6-bce5-3c1ee7ffe019\" (UID: \"147846c0-c5df-45f6-bce5-3c1ee7ffe019\") "
Jul 17 10:11:52.917265 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:52.917243    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/147846c0-c5df-45f6-bce5-3c1ee7ffe019-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "147846c0-c5df-45f6-bce5-3c1ee7ffe019" (UID: "147846c0-c5df-45f6-bce5-3c1ee7ffe019"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:11:53.017856 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:53.017824    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/147846c0-c5df-45f6-bce5-3c1ee7ffe019-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:11:53.618734 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:53.618700    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4" event={"ID":"147846c0-c5df-45f6-bce5-3c1ee7ffe019","Type":"ContainerDied","Data":"457b1410b9168c68dfcb5f42ab6a3e337dc890caa67e0059a32bd09648ee49ae"}
Jul 17 10:11:53.618734 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:53.618725    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"
Jul 17 10:11:53.618981 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:53.618762    2571 scope.go:117] "RemoveContainer" containerID="c18c7c558b091ae104032c781675e684f22df6c99e8b94dc99870909ccab89a1"
Jul 17 10:11:53.659725 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:53.659694    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"]
Jul 17 10:11:53.676187 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:53.676155    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"]
Jul 17 10:11:53.679288 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:53.679262    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-7l2q4"]
Jul 17 10:11:53.761182 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:53.761144    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" path="/var/lib/kubelet/pods/147846c0-c5df-45f6-bce5-3c1ee7ffe019/volumes"
Jul 17 10:11:53.819024 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:53.818998    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"
Jul 17 10:11:53.926006 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:53.925920    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/852364b4-bdda-4adb-a253-9dc14bd02054-maas-api-service-ca\") pod \"852364b4-bdda-4adb-a253-9dc14bd02054\" (UID: \"852364b4-bdda-4adb-a253-9dc14bd02054\") "
Jul 17 10:11:53.926282 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:53.926259    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/852364b4-bdda-4adb-a253-9dc14bd02054-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "852364b4-bdda-4adb-a253-9dc14bd02054" (UID: "852364b4-bdda-4adb-a253-9dc14bd02054"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:11:54.026814 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:54.026779    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/852364b4-bdda-4adb-a253-9dc14bd02054-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:11:54.626101 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:54.626073    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"
Jul 17 10:11:54.626308 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:54.626113    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc" event={"ID":"852364b4-bdda-4adb-a253-9dc14bd02054","Type":"ContainerDied","Data":"b64809ff3d7119ba36ee15ed484435812a6a28066bb515c70b2b8ca0f4be6b15"}
Jul 17 10:11:54.626308 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:54.626181    2571 scope.go:117] "RemoveContainer" containerID="6836a84c6d1c7076fd908ba7c3375b8e8054f15c571225467f73b910a05e304b"
Jul 17 10:11:54.658892 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:54.658857    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"]
Jul 17 10:11:54.668613 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:54.668582    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-9xhpc"]
Jul 17 10:11:55.758494 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:11:55.758435    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="852364b4-bdda-4adb-a253-9dc14bd02054" path="/var/lib/kubelet/pods/852364b4-bdda-4adb-a253-9dc14bd02054/volumes"
Jul 17 10:12:01.752660 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:01.752625    2571 scope.go:117] "RemoveContainer" containerID="662d1f700b82f97c5e5654d8b6db119e88fb861795b972d3eaaca67075c98b9d"
Jul 17 10:12:02.678491 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:02.678393    2571 generic.go:356] "Generic (PLEG): container finished" podID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerID="80ab49044195ce520d6859fa16e73abde7917418bd6210c91720dc1d2d641a44" exitCode=6
Jul 17 10:12:02.678627 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:02.678484    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h" event={"ID":"70c35f0b-a90b-4cd5-8b67-db3d3f17627b","Type":"ContainerDied","Data":"80ab49044195ce520d6859fa16e73abde7917418bd6210c91720dc1d2d641a44"}
Jul 17 10:12:02.678627 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:02.678571    2571 scope.go:117] "RemoveContainer" containerID="662d1f700b82f97c5e5654d8b6db119e88fb861795b972d3eaaca67075c98b9d"
Jul 17 10:12:02.678850 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:02.678827    2571 scope.go:117] "RemoveContainer" containerID="80ab49044195ce520d6859fa16e73abde7917418bd6210c91720dc1d2d641a44"
Jul 17 10:12:02.679114 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:02.679088    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h_odh-ai-gateway-infra(70c35f0b-a90b-4cd5-8b67-db3d3f17627b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b"
Jul 17 10:12:03.730304 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:03.730269    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"]
Jul 17 10:12:03.883244 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:03.883218    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"
Jul 17 10:12:04.025619 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:04.025523    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c35f0b-a90b-4cd5-8b67-db3d3f17627b-maas-api-service-ca\") pod \"70c35f0b-a90b-4cd5-8b67-db3d3f17627b\" (UID: \"70c35f0b-a90b-4cd5-8b67-db3d3f17627b\") "
Jul 17 10:12:04.025940 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:04.025913    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c35f0b-a90b-4cd5-8b67-db3d3f17627b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "70c35f0b-a90b-4cd5-8b67-db3d3f17627b" (UID: "70c35f0b-a90b-4cd5-8b67-db3d3f17627b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:12:04.127000 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:04.126967    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c35f0b-a90b-4cd5-8b67-db3d3f17627b-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:12:04.694636 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:04.694608    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"
Jul 17 10:12:04.694813 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:04.694643    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h" event={"ID":"70c35f0b-a90b-4cd5-8b67-db3d3f17627b","Type":"ContainerDied","Data":"b2f5b8cb86ffe3166f47413d54d8c936ba00e0de0a2ff11cda08ea8022daea60"}
Jul 17 10:12:04.694813 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:04.694706    2571 scope.go:117] "RemoveContainer" containerID="80ab49044195ce520d6859fa16e73abde7917418bd6210c91720dc1d2d641a44"
Jul 17 10:12:04.747267 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:04.747232    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"]
Jul 17 10:12:04.752231 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:04.752209    2571 scope.go:117] "RemoveContainer" containerID="49933a126b03b93b55d71fe3a64a6f2959a2e1c9df0f1c8e2e103c16fd076779"
Jul 17 10:12:04.756499 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:04.756477    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-82p8h"]
Jul 17 10:12:05.702355 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:05.702325    2571 generic.go:356] "Generic (PLEG): container finished" podID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerID="1cf8648fed6069658fa703f6970323ce1daa5a71452dbb733dcb85829cd8a5d5" exitCode=6
Jul 17 10:12:05.702515 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:05.702397    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm" event={"ID":"391e2ee4-3aca-40d0-8b53-73afe862e7c5","Type":"ContainerDied","Data":"1cf8648fed6069658fa703f6970323ce1daa5a71452dbb733dcb85829cd8a5d5"}
Jul 17 10:12:05.702515 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:05.702475    2571 scope.go:117] "RemoveContainer" containerID="49933a126b03b93b55d71fe3a64a6f2959a2e1c9df0f1c8e2e103c16fd076779"
Jul 17 10:12:05.702873 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:05.702856    2571 scope.go:117] "RemoveContainer" containerID="1cf8648fed6069658fa703f6970323ce1daa5a71452dbb733dcb85829cd8a5d5"
Jul 17 10:12:05.703166 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:05.703143    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm_odh-ai-gateway-infra(391e2ee4-3aca-40d0-8b53-73afe862e7c5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5"
Jul 17 10:12:05.759488 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:05.759436    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" path="/var/lib/kubelet/pods/70c35f0b-a90b-4cd5-8b67-db3d3f17627b/volumes"
Jul 17 10:12:06.777722 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:06.777689    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"]
Jul 17 10:12:06.926512 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:06.926488    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"
Jul 17 10:12:07.055234 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:07.055137    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/391e2ee4-3aca-40d0-8b53-73afe862e7c5-maas-api-service-ca\") pod \"391e2ee4-3aca-40d0-8b53-73afe862e7c5\" (UID: \"391e2ee4-3aca-40d0-8b53-73afe862e7c5\") "
Jul 17 10:12:07.055525 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:07.055503    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/391e2ee4-3aca-40d0-8b53-73afe862e7c5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "391e2ee4-3aca-40d0-8b53-73afe862e7c5" (UID: "391e2ee4-3aca-40d0-8b53-73afe862e7c5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:12:07.156098 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:07.156063    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/391e2ee4-3aca-40d0-8b53-73afe862e7c5-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:12:07.718560 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:07.718525    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"
Jul 17 10:12:07.718758 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:07.718577    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm" event={"ID":"391e2ee4-3aca-40d0-8b53-73afe862e7c5","Type":"ContainerDied","Data":"a067be9c6acc43d7c39cbc0f88339156334beab5df50f74fc1f17ab3a563f1fe"}
Jul 17 10:12:07.718758 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:07.718666    2571 scope.go:117] "RemoveContainer" containerID="1cf8648fed6069658fa703f6970323ce1daa5a71452dbb733dcb85829cd8a5d5"
Jul 17 10:12:07.750876 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:07.750841    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"]
Jul 17 10:12:07.759399 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:07.759367    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-t9jzm"]
Jul 17 10:12:09.758064 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:09.758027    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" path="/var/lib/kubelet/pods/391e2ee4-3aca-40d0-8b53-73afe862e7c5/volumes"
Jul 17 10:12:26.222719 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.222679    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"]
Jul 17 10:12:26.223128 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223115    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerName="revoke-keys"
Jul 17 10:12:26.223178 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223131    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerName="revoke-keys"
Jul 17 10:12:26.223178 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223143    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerName="revoke-keys"
Jul 17 10:12:26.223178 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223148    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerName="revoke-keys"
Jul 17 10:12:26.223178 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223155    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerName="revoke-keys"
Jul 17 10:12:26.223178 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223160    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerName="revoke-keys"
Jul 17 10:12:26.223178 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223173    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerName="revoke-keys"
Jul 17 10:12:26.223178 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223178    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223191    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="852364b4-bdda-4adb-a253-9dc14bd02054" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223198    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="852364b4-bdda-4adb-a253-9dc14bd02054" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223206    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223211    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223219    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223224    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223231    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223236    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223244    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="852364b4-bdda-4adb-a253-9dc14bd02054" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223249    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="852364b4-bdda-4adb-a253-9dc14bd02054" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223254    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="852364b4-bdda-4adb-a253-9dc14bd02054" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223259    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="852364b4-bdda-4adb-a253-9dc14bd02054" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223320    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223326    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223332    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223338    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223344    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223349    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223354    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="852364b4-bdda-4adb-a253-9dc14bd02054" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223360    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="852364b4-bdda-4adb-a253-9dc14bd02054" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223365    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="852364b4-bdda-4adb-a253-9dc14bd02054" containerName="revoke-keys"
Jul 17 10:12:26.223387 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.223370    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerName="revoke-keys"
Jul 17 10:12:26.226556 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.226539    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"
Jul 17 10:12:26.231678 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.231658    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:12:26.257670 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.257634    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"]
Jul 17 10:12:26.324409 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.324369    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f989de16-9b6c-4ed2-add2-ee2f9ac1cf09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc\" (UID: \"f989de16-9b6c-4ed2-add2-ee2f9ac1cf09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"
Jul 17 10:12:26.425907 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.425875    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f989de16-9b6c-4ed2-add2-ee2f9ac1cf09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc\" (UID: \"f989de16-9b6c-4ed2-add2-ee2f9ac1cf09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"
Jul 17 10:12:26.426652 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.426631    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f989de16-9b6c-4ed2-add2-ee2f9ac1cf09-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc\" (UID: \"f989de16-9b6c-4ed2-add2-ee2f9ac1cf09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"
Jul 17 10:12:26.537585 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.537506    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"
Jul 17 10:12:26.677031 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.677002    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"]
Jul 17 10:12:26.678951 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:12:26.678911    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf989de16_9b6c_4ed2_add2_ee2f9ac1cf09.slice/crio-8c535aec0c8609aa73c926965a4809ca0605d5b864c34ddf9eb75d6b4a70bbad WatchSource:0}: Error finding container 8c535aec0c8609aa73c926965a4809ca0605d5b864c34ddf9eb75d6b4a70bbad: Status 404 returned error can't find the container with id 8c535aec0c8609aa73c926965a4809ca0605d5b864c34ddf9eb75d6b4a70bbad
Jul 17 10:12:26.835294 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:26.835211    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc" event={"ID":"f989de16-9b6c-4ed2-add2-ee2f9ac1cf09","Type":"ContainerStarted","Data":"8c535aec0c8609aa73c926965a4809ca0605d5b864c34ddf9eb75d6b4a70bbad"}
Jul 17 10:12:27.843013 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:27.842977    2571 generic.go:356] "Generic (PLEG): container finished" podID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerID="f071aa7274bbfeebea636f964fe2f13c2807cafc3d4633b893b9f52bb829113b" exitCode=6
Jul 17 10:12:27.843483 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:27.843065    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc" event={"ID":"f989de16-9b6c-4ed2-add2-ee2f9ac1cf09","Type":"ContainerDied","Data":"f071aa7274bbfeebea636f964fe2f13c2807cafc3d4633b893b9f52bb829113b"}
Jul 17 10:12:27.843483 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:27.843230    2571 scope.go:117] "RemoveContainer" containerID="f071aa7274bbfeebea636f964fe2f13c2807cafc3d4633b893b9f52bb829113b"
Jul 17 10:12:28.851554 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:28.851520    2571 generic.go:356] "Generic (PLEG): container finished" podID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerID="8ee3bc7c0665412c76ea056452bee876ea9ca14808c718ba0313a25b0389b071" exitCode=6
Jul 17 10:12:28.852001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:28.851595    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc" event={"ID":"f989de16-9b6c-4ed2-add2-ee2f9ac1cf09","Type":"ContainerDied","Data":"8ee3bc7c0665412c76ea056452bee876ea9ca14808c718ba0313a25b0389b071"}
Jul 17 10:12:28.852001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:28.851649    2571 scope.go:117] "RemoveContainer" containerID="f071aa7274bbfeebea636f964fe2f13c2807cafc3d4633b893b9f52bb829113b"
Jul 17 10:12:28.852001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:28.851780    2571 scope.go:117] "RemoveContainer" containerID="8ee3bc7c0665412c76ea056452bee876ea9ca14808c718ba0313a25b0389b071"
Jul 17 10:12:28.852001 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:28.851968    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc_odh-ai-gateway-infra(f989de16-9b6c-4ed2-add2-ee2f9ac1cf09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09"
Jul 17 10:12:29.859928 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:29.859900    2571 scope.go:117] "RemoveContainer" containerID="8ee3bc7c0665412c76ea056452bee876ea9ca14808c718ba0313a25b0389b071"
Jul 17 10:12:29.860401 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:29.860129    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc_odh-ai-gateway-infra(f989de16-9b6c-4ed2-add2-ee2f9ac1cf09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09"
Jul 17 10:12:33.737547 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.737499    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"]
Jul 17 10:12:33.737973 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.737961    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerName="revoke-keys"
Jul 17 10:12:33.737973 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.737975    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="391e2ee4-3aca-40d0-8b53-73afe862e7c5" containerName="revoke-keys"
Jul 17 10:12:33.738058 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.737987    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerName="revoke-keys"
Jul 17 10:12:33.738058 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.737992    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerName="revoke-keys"
Jul 17 10:12:33.738145 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.738059    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="147846c0-c5df-45f6-bce5-3c1ee7ffe019" containerName="revoke-keys"
Jul 17 10:12:33.738145 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.738067    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c35f0b-a90b-4cd5-8b67-db3d3f17627b" containerName="revoke-keys"
Jul 17 10:12:33.750300 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.750262    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"
Jul 17 10:12:33.750601 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.750565    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"]
Jul 17 10:12:33.790940 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.790908    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ef7c03-0280-4190-8a1a-43d952f126ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5\" (UID: \"91ef7c03-0280-4190-8a1a-43d952f126ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"
Jul 17 10:12:33.891638 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.891604    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ef7c03-0280-4190-8a1a-43d952f126ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5\" (UID: \"91ef7c03-0280-4190-8a1a-43d952f126ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"
Jul 17 10:12:33.892311 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:33.892290    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ef7c03-0280-4190-8a1a-43d952f126ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5\" (UID: \"91ef7c03-0280-4190-8a1a-43d952f126ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"
Jul 17 10:12:34.061776 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:34.061694    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"
Jul 17 10:12:34.210408 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:34.210374    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"]
Jul 17 10:12:34.211225 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:12:34.211199    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ef7c03_0280_4190_8a1a_43d952f126ba.slice/crio-27a3ad64ecf4e934b04cc9e62c6a592dbae7ad95289eb759270517fc43fa6ed7 WatchSource:0}: Error finding container 27a3ad64ecf4e934b04cc9e62c6a592dbae7ad95289eb759270517fc43fa6ed7: Status 404 returned error can't find the container with id 27a3ad64ecf4e934b04cc9e62c6a592dbae7ad95289eb759270517fc43fa6ed7
Jul 17 10:12:34.896495 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:34.896430    2571 generic.go:356] "Generic (PLEG): container finished" podID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerID="2e97007e9ad2b8bd799db1860eb827fcb09b1a2d8d0175fa18f2fc5e34663c8e" exitCode=6
Jul 17 10:12:34.896860 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:34.896506    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5" event={"ID":"91ef7c03-0280-4190-8a1a-43d952f126ba","Type":"ContainerDied","Data":"2e97007e9ad2b8bd799db1860eb827fcb09b1a2d8d0175fa18f2fc5e34663c8e"}
Jul 17 10:12:34.896860 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:34.896568    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5" event={"ID":"91ef7c03-0280-4190-8a1a-43d952f126ba","Type":"ContainerStarted","Data":"27a3ad64ecf4e934b04cc9e62c6a592dbae7ad95289eb759270517fc43fa6ed7"}
Jul 17 10:12:34.896860 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:34.896805    2571 scope.go:117] "RemoveContainer" containerID="2e97007e9ad2b8bd799db1860eb827fcb09b1a2d8d0175fa18f2fc5e34663c8e"
Jul 17 10:12:35.903936 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:35.903902    2571 generic.go:356] "Generic (PLEG): container finished" podID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerID="c71fd42dd3b5abdc661b303987dc0a5508342b4f698a5e18ef139a79f87225b1" exitCode=6
Jul 17 10:12:35.904370 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:35.903985    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5" event={"ID":"91ef7c03-0280-4190-8a1a-43d952f126ba","Type":"ContainerDied","Data":"c71fd42dd3b5abdc661b303987dc0a5508342b4f698a5e18ef139a79f87225b1"}
Jul 17 10:12:35.904370 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:35.904041    2571 scope.go:117] "RemoveContainer" containerID="2e97007e9ad2b8bd799db1860eb827fcb09b1a2d8d0175fa18f2fc5e34663c8e"
Jul 17 10:12:35.904370 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:35.904273    2571 scope.go:117] "RemoveContainer" containerID="c71fd42dd3b5abdc661b303987dc0a5508342b4f698a5e18ef139a79f87225b1"
Jul 17 10:12:35.904523 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:35.904504    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5_odh-ai-gateway-infra(91ef7c03-0280-4190-8a1a-43d952f126ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba"
Jul 17 10:12:36.913022 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:36.912999    2571 scope.go:117] "RemoveContainer" containerID="c71fd42dd3b5abdc661b303987dc0a5508342b4f698a5e18ef139a79f87225b1"
Jul 17 10:12:36.913368 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:36.913193    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5_odh-ai-gateway-infra(91ef7c03-0280-4190-8a1a-43d952f126ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba"
Jul 17 10:12:42.752761 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:42.752721    2571 scope.go:117] "RemoveContainer" containerID="8ee3bc7c0665412c76ea056452bee876ea9ca14808c718ba0313a25b0389b071"
Jul 17 10:12:43.956555 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:43.956518    2571 generic.go:356] "Generic (PLEG): container finished" podID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerID="24ba945eb025db60c3ccabda2c0b70d26f62b82b262081c8afd2d491042126fd" exitCode=6
Jul 17 10:12:43.956985 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:43.956592    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc" event={"ID":"f989de16-9b6c-4ed2-add2-ee2f9ac1cf09","Type":"ContainerDied","Data":"24ba945eb025db60c3ccabda2c0b70d26f62b82b262081c8afd2d491042126fd"}
Jul 17 10:12:43.956985 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:43.956649    2571 scope.go:117] "RemoveContainer" containerID="8ee3bc7c0665412c76ea056452bee876ea9ca14808c718ba0313a25b0389b071"
Jul 17 10:12:43.956985 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:43.956902    2571 scope.go:117] "RemoveContainer" containerID="24ba945eb025db60c3ccabda2c0b70d26f62b82b262081c8afd2d491042126fd"
Jul 17 10:12:43.957171 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:43.957117    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc_odh-ai-gateway-infra(f989de16-9b6c-4ed2-add2-ee2f9ac1cf09)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09"
Jul 17 10:12:44.792236 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:44.792197    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"]
Jul 17 10:12:44.797913 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:44.797885    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"
Jul 17 10:12:44.804151 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:44.804121    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"]
Jul 17 10:12:44.889940 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:44.889904    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/293f5255-02b5-492d-91be-c9389a0bf2c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-h257t\" (UID: \"293f5255-02b5-492d-91be-c9389a0bf2c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"
Jul 17 10:12:44.988932 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:44.988889    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"]
Jul 17 10:12:44.990982 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:44.990957    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/293f5255-02b5-492d-91be-c9389a0bf2c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-h257t\" (UID: \"293f5255-02b5-492d-91be-c9389a0bf2c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"
Jul 17 10:12:44.991593 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:44.991570    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/293f5255-02b5-492d-91be-c9389a0bf2c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-h257t\" (UID: \"293f5255-02b5-492d-91be-c9389a0bf2c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"
Jul 17 10:12:45.112486 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.112441    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"
Jul 17 10:12:45.146777 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.146755    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"
Jul 17 10:12:45.193476 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.193058    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f989de16-9b6c-4ed2-add2-ee2f9ac1cf09-maas-api-service-ca\") pod \"f989de16-9b6c-4ed2-add2-ee2f9ac1cf09\" (UID: \"f989de16-9b6c-4ed2-add2-ee2f9ac1cf09\") "
Jul 17 10:12:45.193657 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.193490    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f989de16-9b6c-4ed2-add2-ee2f9ac1cf09-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" (UID: "f989de16-9b6c-4ed2-add2-ee2f9ac1cf09"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:12:45.294234 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.294152    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f989de16-9b6c-4ed2-add2-ee2f9ac1cf09-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:12:45.465835 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.465799    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"]
Jul 17 10:12:45.466838 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:12:45.466809    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod293f5255_02b5_492d_91be_c9389a0bf2c0.slice/crio-4561c8135bc101b813d073838b1ed2caa10c8af76b351c335b1bf2d717f77162 WatchSource:0}: Error finding container 4561c8135bc101b813d073838b1ed2caa10c8af76b351c335b1bf2d717f77162: Status 404 returned error can't find the container with id 4561c8135bc101b813d073838b1ed2caa10c8af76b351c335b1bf2d717f77162
Jul 17 10:12:45.915627 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.915546    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"]
Jul 17 10:12:45.916005 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.915992    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerName="revoke-keys"
Jul 17 10:12:45.916054 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.916007    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerName="revoke-keys"
Jul 17 10:12:45.916054 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.916019    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerName="revoke-keys"
Jul 17 10:12:45.916054 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.916024    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerName="revoke-keys"
Jul 17 10:12:45.916153 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.916087    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerName="revoke-keys"
Jul 17 10:12:45.916153 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.916096    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerName="revoke-keys"
Jul 17 10:12:45.916153 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.916104    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerName="revoke-keys"
Jul 17 10:12:45.919267 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.919250    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"
Jul 17 10:12:45.929356 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.929327    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"]
Jul 17 10:12:45.971491 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.971445    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"
Jul 17 10:12:45.971675 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.971455    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc" event={"ID":"f989de16-9b6c-4ed2-add2-ee2f9ac1cf09","Type":"ContainerDied","Data":"8c535aec0c8609aa73c926965a4809ca0605d5b864c34ddf9eb75d6b4a70bbad"}
Jul 17 10:12:45.971675 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.971628    2571 scope.go:117] "RemoveContainer" containerID="24ba945eb025db60c3ccabda2c0b70d26f62b82b262081c8afd2d491042126fd"
Jul 17 10:12:45.973323 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.973295    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t" event={"ID":"293f5255-02b5-492d-91be-c9389a0bf2c0","Type":"ContainerStarted","Data":"7acccc4c1f4710aef99e7cfcfdb853ca06cb69575369a6ef9b070e752cf42d42"}
Jul 17 10:12:45.973440 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.973348    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t" event={"ID":"293f5255-02b5-492d-91be-c9389a0bf2c0","Type":"ContainerStarted","Data":"4561c8135bc101b813d073838b1ed2caa10c8af76b351c335b1bf2d717f77162"}
Jul 17 10:12:45.990853 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.990831    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"]
Jul 17 10:12:45.993702 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:45.993677    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-5r5gc"]
Jul 17 10:12:46.002057 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:46.002033    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89e8a616-d36f-46b3-a2d6-58bb3d02923d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws\" (UID: \"89e8a616-d36f-46b3-a2d6-58bb3d02923d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"
Jul 17 10:12:46.010899 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:46.010853    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t" podStartSLOduration=2.01083854 podStartE2EDuration="2.01083854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:12:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:12:46.00950772 +0000 UTC m=+3250.074328442" watchObservedRunningTime="2026-07-17 10:12:46.01083854 +0000 UTC m=+3250.075659271"
Jul 17 10:12:46.103339 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:46.103289    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89e8a616-d36f-46b3-a2d6-58bb3d02923d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws\" (UID: \"89e8a616-d36f-46b3-a2d6-58bb3d02923d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"
Jul 17 10:12:46.103928 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:46.103906    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89e8a616-d36f-46b3-a2d6-58bb3d02923d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws\" (UID: \"89e8a616-d36f-46b3-a2d6-58bb3d02923d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"
Jul 17 10:12:46.230917 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:46.230883    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"
Jul 17 10:12:46.384969 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:46.384936    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"]
Jul 17 10:12:46.385857 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:12:46.385830    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89e8a616_d36f_46b3_a2d6_58bb3d02923d.slice/crio-965123e3f99a184d1d3205b85feb01e77cae18878703cbff7f482cc06e6f99fb WatchSource:0}: Error finding container 965123e3f99a184d1d3205b85feb01e77cae18878703cbff7f482cc06e6f99fb: Status 404 returned error can't find the container with id 965123e3f99a184d1d3205b85feb01e77cae18878703cbff7f482cc06e6f99fb
Jul 17 10:12:46.981125 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:46.981096    2571 generic.go:356] "Generic (PLEG): container finished" podID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerID="7acccc4c1f4710aef99e7cfcfdb853ca06cb69575369a6ef9b070e752cf42d42" exitCode=6
Jul 17 10:12:46.981332 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:46.981163    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t" event={"ID":"293f5255-02b5-492d-91be-c9389a0bf2c0","Type":"ContainerDied","Data":"7acccc4c1f4710aef99e7cfcfdb853ca06cb69575369a6ef9b070e752cf42d42"}
Jul 17 10:12:46.981421 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:46.981406    2571 scope.go:117] "RemoveContainer" containerID="7acccc4c1f4710aef99e7cfcfdb853ca06cb69575369a6ef9b070e752cf42d42"
Jul 17 10:12:46.982885 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:46.982857    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws" event={"ID":"89e8a616-d36f-46b3-a2d6-58bb3d02923d","Type":"ContainerStarted","Data":"1e1aa111eba3b17557786d67c1437ac408855e5291878a5608156efb5d5f3663"}
Jul 17 10:12:46.983003 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:46.982911    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws" event={"ID":"89e8a616-d36f-46b3-a2d6-58bb3d02923d","Type":"ContainerStarted","Data":"965123e3f99a184d1d3205b85feb01e77cae18878703cbff7f482cc06e6f99fb"}
Jul 17 10:12:47.058757 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:47.058705    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws" podStartSLOduration=2.058689338 podStartE2EDuration="2.058689338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:12:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:12:47.057575596 +0000 UTC m=+3251.122396319" watchObservedRunningTime="2026-07-17 10:12:47.058689338 +0000 UTC m=+3251.123510060"
Jul 17 10:12:47.762658 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:47.760210    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" path="/var/lib/kubelet/pods/f989de16-9b6c-4ed2-add2-ee2f9ac1cf09/volumes"
Jul 17 10:12:47.992074 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:47.992039    2571 generic.go:356] "Generic (PLEG): container finished" podID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerID="1e1aa111eba3b17557786d67c1437ac408855e5291878a5608156efb5d5f3663" exitCode=6
Jul 17 10:12:47.992265 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:47.992134    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws" event={"ID":"89e8a616-d36f-46b3-a2d6-58bb3d02923d","Type":"ContainerDied","Data":"1e1aa111eba3b17557786d67c1437ac408855e5291878a5608156efb5d5f3663"}
Jul 17 10:12:47.992329 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:47.992290    2571 scope.go:117] "RemoveContainer" containerID="1e1aa111eba3b17557786d67c1437ac408855e5291878a5608156efb5d5f3663"
Jul 17 10:12:47.994073 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:47.994053    2571 generic.go:356] "Generic (PLEG): container finished" podID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerID="6341de5b81c4bc29ef3b6a46da6596806dca1c680efca3e64637a52db020328f" exitCode=6
Jul 17 10:12:47.994206 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:47.994085    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t" event={"ID":"293f5255-02b5-492d-91be-c9389a0bf2c0","Type":"ContainerDied","Data":"6341de5b81c4bc29ef3b6a46da6596806dca1c680efca3e64637a52db020328f"}
Jul 17 10:12:47.994206 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:47.994128    2571 scope.go:117] "RemoveContainer" containerID="7acccc4c1f4710aef99e7cfcfdb853ca06cb69575369a6ef9b070e752cf42d42"
Jul 17 10:12:47.994378 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:47.994355    2571 scope.go:117] "RemoveContainer" containerID="6341de5b81c4bc29ef3b6a46da6596806dca1c680efca3e64637a52db020328f"
Jul 17 10:12:47.994752 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:47.994620    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-h257t_odh-ai-gateway-infra(293f5255-02b5-492d-91be-c9389a0bf2c0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0"
Jul 17 10:12:49.002698 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:49.002670    2571 scope.go:117] "RemoveContainer" containerID="6341de5b81c4bc29ef3b6a46da6596806dca1c680efca3e64637a52db020328f"
Jul 17 10:12:49.003149 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:49.002922    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-h257t_odh-ai-gateway-infra(293f5255-02b5-492d-91be-c9389a0bf2c0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0"
Jul 17 10:12:49.004285 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:49.004256    2571 generic.go:356] "Generic (PLEG): container finished" podID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerID="45b5d06871ab70521baa1cf3339fa5424ecf9f5c6235e0bc0e8f550dc2e150cb" exitCode=6
Jul 17 10:12:49.004390 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:49.004340    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws" event={"ID":"89e8a616-d36f-46b3-a2d6-58bb3d02923d","Type":"ContainerDied","Data":"45b5d06871ab70521baa1cf3339fa5424ecf9f5c6235e0bc0e8f550dc2e150cb"}
Jul 17 10:12:49.004487 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:49.004397    2571 scope.go:117] "RemoveContainer" containerID="1e1aa111eba3b17557786d67c1437ac408855e5291878a5608156efb5d5f3663"
Jul 17 10:12:49.004543 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:49.004526    2571 scope.go:117] "RemoveContainer" containerID="45b5d06871ab70521baa1cf3339fa5424ecf9f5c6235e0bc0e8f550dc2e150cb"
Jul 17 10:12:49.004749 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:49.004728    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws_odh-ai-gateway-infra(89e8a616-d36f-46b3-a2d6-58bb3d02923d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d"
Jul 17 10:12:49.753089 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:49.753059    2571 scope.go:117] "RemoveContainer" containerID="c71fd42dd3b5abdc661b303987dc0a5508342b4f698a5e18ef139a79f87225b1"
Jul 17 10:12:50.012526 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:50.012421    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5" event={"ID":"91ef7c03-0280-4190-8a1a-43d952f126ba","Type":"ContainerStarted","Data":"ee8621d292a5dd2e3984ad1665b4ed7761b6a03557ec4f9c5cd2a71d4af2bba7"}
Jul 17 10:12:50.014331 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:50.014310    2571 scope.go:117] "RemoveContainer" containerID="45b5d06871ab70521baa1cf3339fa5424ecf9f5c6235e0bc0e8f550dc2e150cb"
Jul 17 10:12:50.014546 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:50.014526    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws_odh-ai-gateway-infra(89e8a616-d36f-46b3-a2d6-58bb3d02923d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d"
Jul 17 10:12:50.035549 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:50.035503    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5" podStartSLOduration=17.035486104 podStartE2EDuration="17.035486104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:12:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:12:50.03329765 +0000 UTC m=+3254.098118379" watchObservedRunningTime="2026-07-17 10:12:50.035486104 +0000 UTC m=+3254.100306826"
Jul 17 10:12:51.022710 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:51.022669    2571 generic.go:356] "Generic (PLEG): container finished" podID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerID="ee8621d292a5dd2e3984ad1665b4ed7761b6a03557ec4f9c5cd2a71d4af2bba7" exitCode=6
Jul 17 10:12:51.023157 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:51.022738    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5" event={"ID":"91ef7c03-0280-4190-8a1a-43d952f126ba","Type":"ContainerDied","Data":"ee8621d292a5dd2e3984ad1665b4ed7761b6a03557ec4f9c5cd2a71d4af2bba7"}
Jul 17 10:12:51.023157 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:51.022803    2571 scope.go:117] "RemoveContainer" containerID="c71fd42dd3b5abdc661b303987dc0a5508342b4f698a5e18ef139a79f87225b1"
Jul 17 10:12:51.023157 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:51.023083    2571 scope.go:117] "RemoveContainer" containerID="ee8621d292a5dd2e3984ad1665b4ed7761b6a03557ec4f9c5cd2a71d4af2bba7"
Jul 17 10:12:51.023370 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:12:51.023346    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5_odh-ai-gateway-infra(91ef7c03-0280-4190-8a1a-43d952f126ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba"
Jul 17 10:12:51.062774 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:51.062736    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"]
Jul 17 10:12:52.179108 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:52.179079    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"
Jul 17 10:12:52.264323 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:52.264290    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ef7c03-0280-4190-8a1a-43d952f126ba-maas-api-service-ca\") pod \"91ef7c03-0280-4190-8a1a-43d952f126ba\" (UID: \"91ef7c03-0280-4190-8a1a-43d952f126ba\") "
Jul 17 10:12:52.264737 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:52.264711    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91ef7c03-0280-4190-8a1a-43d952f126ba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "91ef7c03-0280-4190-8a1a-43d952f126ba" (UID: "91ef7c03-0280-4190-8a1a-43d952f126ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:12:52.366056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:52.365959    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/91ef7c03-0280-4190-8a1a-43d952f126ba-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:12:53.037842 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:53.037811    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"
Jul 17 10:12:53.038041 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:53.037845    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5" event={"ID":"91ef7c03-0280-4190-8a1a-43d952f126ba","Type":"ContainerDied","Data":"27a3ad64ecf4e934b04cc9e62c6a592dbae7ad95289eb759270517fc43fa6ed7"}
Jul 17 10:12:53.038041 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:53.037906    2571 scope.go:117] "RemoveContainer" containerID="ee8621d292a5dd2e3984ad1665b4ed7761b6a03557ec4f9c5cd2a71d4af2bba7"
Jul 17 10:12:53.070815 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:53.070782    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"]
Jul 17 10:12:53.077993 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:53.077969    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-qtpr5"]
Jul 17 10:12:53.758536 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:53.758505    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba" path="/var/lib/kubelet/pods/91ef7c03-0280-4190-8a1a-43d952f126ba/volumes"
Jul 17 10:12:57.436788 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.436746    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"]
Jul 17 10:12:57.437495 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.437450    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerName="revoke-keys"
Jul 17 10:12:57.437495 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.437494    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f989de16-9b6c-4ed2-add2-ee2f9ac1cf09" containerName="revoke-keys"
Jul 17 10:12:57.437632 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.437510    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerName="revoke-keys"
Jul 17 10:12:57.437632 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.437520    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerName="revoke-keys"
Jul 17 10:12:57.437632 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.437539    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerName="revoke-keys"
Jul 17 10:12:57.437632 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.437548    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerName="revoke-keys"
Jul 17 10:12:57.437855 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.437697    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerName="revoke-keys"
Jul 17 10:12:57.437855 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.437711    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerName="revoke-keys"
Jul 17 10:12:57.437855 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.437721    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerName="revoke-keys"
Jul 17 10:12:57.442789 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.442768    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.448770 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.448747    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"default-dockercfg-q2mhr\""
Jul 17 10:12:57.449354 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.449330    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"openshift-service-ca.crt\""
Jul 17 10:12:57.449534 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.449503    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"sub-test-model-8ae1eb-kserve-self-signed-certs\""
Jul 17 10:12:57.450063 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.450045    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8ae1eb\"/\"kube-root-ca.crt\""
Jul 17 10:12:57.463977 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.463361    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"]
Jul 17 10:12:57.514618 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.514584    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-home\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.514800 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.514648    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-model-cache\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.514800 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.514728    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mzl4\" (UniqueName: \"kubernetes.io/projected/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-kube-api-access-7mzl4\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.514800 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.514769    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-dshm\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.514800 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.514795    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tmp-dir\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.515003 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.514842    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tls-certs\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.615693 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.615654    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-home\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.615877 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.615721    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-model-cache\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.615877 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.615750    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mzl4\" (UniqueName: \"kubernetes.io/projected/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-kube-api-access-7mzl4\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.615877 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.615779    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-dshm\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.615877 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.615794    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tmp-dir\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.615877 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.615830    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tls-certs\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.616151 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.616129    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-home\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.616194 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.616161    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-model-cache\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.616289 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.616263    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tmp-dir\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.618161 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.618143    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-dshm\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.618445 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.618429    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tls-certs\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.630563 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.630538    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mzl4\" (UniqueName: \"kubernetes.io/projected/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-kube-api-access-7mzl4\") pod \"sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") " pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.752017 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.751979    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:12:57.902495 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:57.902431    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"]
Jul 17 10:12:57.904183 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:12:57.904156    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7e8ec60_9975_4d24_890a_2ba0a6dcc114.slice/crio-58ad706ead42277fb8c9941d0b3d78acb2beb394204b000657aba68b22ca9c13 WatchSource:0}: Error finding container 58ad706ead42277fb8c9941d0b3d78acb2beb394204b000657aba68b22ca9c13: Status 404 returned error can't find the container with id 58ad706ead42277fb8c9941d0b3d78acb2beb394204b000657aba68b22ca9c13
Jul 17 10:12:58.069746 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:58.069670    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m" event={"ID":"e7e8ec60-9975-4d24-890a-2ba0a6dcc114","Type":"ContainerStarted","Data":"b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95"}
Jul 17 10:12:58.069746 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:58.069708    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m" event={"ID":"e7e8ec60-9975-4d24-890a-2ba0a6dcc114","Type":"ContainerStarted","Data":"58ad706ead42277fb8c9941d0b3d78acb2beb394204b000657aba68b22ca9c13"}
Jul 17 10:12:58.092781 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:12:58.092726    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m" podStartSLOduration=1.092711566 podStartE2EDuration="1.092711566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:12:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:12:58.089763631 +0000 UTC m=+3262.154584353" watchObservedRunningTime="2026-07-17 10:12:58.092711566 +0000 UTC m=+3262.157532296"
Jul 17 10:13:00.752773 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:00.752734    2571 scope.go:117] "RemoveContainer" containerID="6341de5b81c4bc29ef3b6a46da6596806dca1c680efca3e64637a52db020328f"
Jul 17 10:13:01.091204 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:01.091161    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t" event={"ID":"293f5255-02b5-492d-91be-c9389a0bf2c0","Type":"ContainerStarted","Data":"99e0fb1c5c99f32e11d8dcacff670dc5db94332ce59f07ad0d9b1f3a9ef6f15e"}
Jul 17 10:13:02.098440 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:02.098408    2571 generic.go:356] "Generic (PLEG): container finished" podID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerID="99e0fb1c5c99f32e11d8dcacff670dc5db94332ce59f07ad0d9b1f3a9ef6f15e" exitCode=6
Jul 17 10:13:02.098902 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:02.098499    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t" event={"ID":"293f5255-02b5-492d-91be-c9389a0bf2c0","Type":"ContainerDied","Data":"99e0fb1c5c99f32e11d8dcacff670dc5db94332ce59f07ad0d9b1f3a9ef6f15e"}
Jul 17 10:13:02.098902 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:02.098558    2571 scope.go:117] "RemoveContainer" containerID="6341de5b81c4bc29ef3b6a46da6596806dca1c680efca3e64637a52db020328f"
Jul 17 10:13:02.098902 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:02.098843    2571 scope.go:117] "RemoveContainer" containerID="99e0fb1c5c99f32e11d8dcacff670dc5db94332ce59f07ad0d9b1f3a9ef6f15e"
Jul 17 10:13:02.099085 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:02.099062    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-h257t_odh-ai-gateway-infra(293f5255-02b5-492d-91be-c9389a0bf2c0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0"
Jul 17 10:13:02.133468 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:02.133442    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"]
Jul 17 10:13:03.241315 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:03.241290    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"
Jul 17 10:13:03.369680 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:03.369590    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/293f5255-02b5-492d-91be-c9389a0bf2c0-maas-api-service-ca\") pod \"293f5255-02b5-492d-91be-c9389a0bf2c0\" (UID: \"293f5255-02b5-492d-91be-c9389a0bf2c0\") "
Jul 17 10:13:03.369954 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:03.369930    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/293f5255-02b5-492d-91be-c9389a0bf2c0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "293f5255-02b5-492d-91be-c9389a0bf2c0" (UID: "293f5255-02b5-492d-91be-c9389a0bf2c0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:13:03.470966 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:03.470923    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/293f5255-02b5-492d-91be-c9389a0bf2c0-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:13:04.113305 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:04.113217    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t" event={"ID":"293f5255-02b5-492d-91be-c9389a0bf2c0","Type":"ContainerDied","Data":"4561c8135bc101b813d073838b1ed2caa10c8af76b351c335b1bf2d717f77162"}
Jul 17 10:13:04.113305 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:04.113252    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"
Jul 17 10:13:04.113305 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:04.113284    2571 scope.go:117] "RemoveContainer" containerID="99e0fb1c5c99f32e11d8dcacff670dc5db94332ce59f07ad0d9b1f3a9ef6f15e"
Jul 17 10:13:04.147774 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:04.147730    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"]
Jul 17 10:13:04.157888 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:04.157852    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-h257t"]
Jul 17 10:13:04.752622 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:04.752591    2571 scope.go:117] "RemoveContainer" containerID="45b5d06871ab70521baa1cf3339fa5424ecf9f5c6235e0bc0e8f550dc2e150cb"
Jul 17 10:13:05.122378 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:05.122344    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws" event={"ID":"89e8a616-d36f-46b3-a2d6-58bb3d02923d","Type":"ContainerStarted","Data":"886dc7b444fc1877b2c95f44858ec00f0d48e7cb0185853a890b1934aa2a7c76"}
Jul 17 10:13:05.758449 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:05.758411    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0" path="/var/lib/kubelet/pods/293f5255-02b5-492d-91be-c9389a0bf2c0/volumes"
Jul 17 10:13:06.130103 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:06.130015    2571 generic.go:356] "Generic (PLEG): container finished" podID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerID="886dc7b444fc1877b2c95f44858ec00f0d48e7cb0185853a890b1934aa2a7c76" exitCode=6
Jul 17 10:13:06.130103 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:06.130058    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws" event={"ID":"89e8a616-d36f-46b3-a2d6-58bb3d02923d","Type":"ContainerDied","Data":"886dc7b444fc1877b2c95f44858ec00f0d48e7cb0185853a890b1934aa2a7c76"}
Jul 17 10:13:06.130298 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:06.130121    2571 scope.go:117] "RemoveContainer" containerID="45b5d06871ab70521baa1cf3339fa5424ecf9f5c6235e0bc0e8f550dc2e150cb"
Jul 17 10:13:06.130392 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:06.130372    2571 scope.go:117] "RemoveContainer" containerID="886dc7b444fc1877b2c95f44858ec00f0d48e7cb0185853a890b1934aa2a7c76"
Jul 17 10:13:06.130671 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:06.130646    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws_odh-ai-gateway-infra(89e8a616-d36f-46b3-a2d6-58bb3d02923d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d"
Jul 17 10:13:06.168884 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:06.168849    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"]
Jul 17 10:13:07.278926 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:07.278901    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"
Jul 17 10:13:07.409251 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:07.409166    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89e8a616-d36f-46b3-a2d6-58bb3d02923d-maas-api-service-ca\") pod \"89e8a616-d36f-46b3-a2d6-58bb3d02923d\" (UID: \"89e8a616-d36f-46b3-a2d6-58bb3d02923d\") "
Jul 17 10:13:07.409608 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:07.409586    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89e8a616-d36f-46b3-a2d6-58bb3d02923d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "89e8a616-d36f-46b3-a2d6-58bb3d02923d" (UID: "89e8a616-d36f-46b3-a2d6-58bb3d02923d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:13:07.510455 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:07.510425    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89e8a616-d36f-46b3-a2d6-58bb3d02923d-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:13:07.759597 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:07.759570    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:13:07.759771 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:07.759641    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:13:07.765372 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:07.765349    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:13:08.145043 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:08.144958    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"
Jul 17 10:13:08.145043 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:08.145001    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws" event={"ID":"89e8a616-d36f-46b3-a2d6-58bb3d02923d","Type":"ContainerDied","Data":"965123e3f99a184d1d3205b85feb01e77cae18878703cbff7f482cc06e6f99fb"}
Jul 17 10:13:08.145286 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:08.145083    2571 scope.go:117] "RemoveContainer" containerID="886dc7b444fc1877b2c95f44858ec00f0d48e7cb0185853a890b1934aa2a7c76"
Jul 17 10:13:08.157274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:08.157245    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:13:08.165626 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:08.165602    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"]
Jul 17 10:13:08.173908 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:08.173886    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-cbmws"]
Jul 17 10:13:09.762368 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:09.762327    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" path="/var/lib/kubelet/pods/89e8a616-d36f-46b3-a2d6-58bb3d02923d/volumes"
Jul 17 10:13:16.279541 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.279504    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"]
Jul 17 10:13:16.279951 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.279937    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerName="revoke-keys"
Jul 17 10:13:16.279951 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.279952    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerName="revoke-keys"
Jul 17 10:13:16.280032 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.279961    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerName="revoke-keys"
Jul 17 10:13:16.280032 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.279968    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="91ef7c03-0280-4190-8a1a-43d952f126ba" containerName="revoke-keys"
Jul 17 10:13:16.280032 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.279977    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerName="revoke-keys"
Jul 17 10:13:16.280032 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.279982    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerName="revoke-keys"
Jul 17 10:13:16.280032 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.279999    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerName="revoke-keys"
Jul 17 10:13:16.280032 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.280004    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerName="revoke-keys"
Jul 17 10:13:16.280032 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.280011    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerName="revoke-keys"
Jul 17 10:13:16.280032 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.280016    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerName="revoke-keys"
Jul 17 10:13:16.280032 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.280022    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerName="revoke-keys"
Jul 17 10:13:16.280032 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.280027    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerName="revoke-keys"
Jul 17 10:13:16.280327 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.280092    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerName="revoke-keys"
Jul 17 10:13:16.280327 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.280102    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerName="revoke-keys"
Jul 17 10:13:16.280327 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.280108    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerName="revoke-keys"
Jul 17 10:13:16.280327 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.280116    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="293f5255-02b5-492d-91be-c9389a0bf2c0" containerName="revoke-keys"
Jul 17 10:13:16.280327 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.280122    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerName="revoke-keys"
Jul 17 10:13:16.284596 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.284580    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"
Jul 17 10:13:16.288336 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.288315    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:13:16.298301 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.298270    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"]
Jul 17 10:13:16.375019 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.374981    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd0a31d2-f50a-41b0-8c1d-81f8d797f007-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n\" (UID: \"fd0a31d2-f50a-41b0-8c1d-81f8d797f007\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"
Jul 17 10:13:16.475971 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.475935    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd0a31d2-f50a-41b0-8c1d-81f8d797f007-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n\" (UID: \"fd0a31d2-f50a-41b0-8c1d-81f8d797f007\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"
Jul 17 10:13:16.476607 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.476585    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd0a31d2-f50a-41b0-8c1d-81f8d797f007-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n\" (UID: \"fd0a31d2-f50a-41b0-8c1d-81f8d797f007\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"
Jul 17 10:13:16.595538 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.595409    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"
Jul 17 10:13:16.750223 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:16.750177    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"]
Jul 17 10:13:16.750367 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:13:16.750294    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0a31d2_f50a_41b0_8c1d_81f8d797f007.slice/crio-bdcd51d756e7e31c6b69c06a1c3419bf71cd017e447c185e0a425fc924747b10 WatchSource:0}: Error finding container bdcd51d756e7e31c6b69c06a1c3419bf71cd017e447c185e0a425fc924747b10: Status 404 returned error can't find the container with id bdcd51d756e7e31c6b69c06a1c3419bf71cd017e447c185e0a425fc924747b10
Jul 17 10:13:17.201006 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:17.200918    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n" event={"ID":"fd0a31d2-f50a-41b0-8c1d-81f8d797f007","Type":"ContainerStarted","Data":"0c82f359546d036efd40189073e668e02c1917b2b4605dfed3cc297e01f69108"}
Jul 17 10:13:17.201006 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:17.200968    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n" event={"ID":"fd0a31d2-f50a-41b0-8c1d-81f8d797f007","Type":"ContainerStarted","Data":"bdcd51d756e7e31c6b69c06a1c3419bf71cd017e447c185e0a425fc924747b10"}
Jul 17 10:13:17.243023 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:17.242974    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n" podStartSLOduration=1.242958741 podStartE2EDuration="1.242958741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:13:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:13:17.241624574 +0000 UTC m=+3281.306445297" watchObservedRunningTime="2026-07-17 10:13:17.242958741 +0000 UTC m=+3281.307779462"
Jul 17 10:13:18.209037 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:18.209001    2571 generic.go:356] "Generic (PLEG): container finished" podID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerID="0c82f359546d036efd40189073e668e02c1917b2b4605dfed3cc297e01f69108" exitCode=6
Jul 17 10:13:18.209488 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:18.209094    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n" event={"ID":"fd0a31d2-f50a-41b0-8c1d-81f8d797f007","Type":"ContainerDied","Data":"0c82f359546d036efd40189073e668e02c1917b2b4605dfed3cc297e01f69108"}
Jul 17 10:13:18.209488 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:18.209283    2571 scope.go:117] "RemoveContainer" containerID="0c82f359546d036efd40189073e668e02c1917b2b4605dfed3cc297e01f69108"
Jul 17 10:13:19.217227 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:19.217192    2571 generic.go:356] "Generic (PLEG): container finished" podID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerID="72d13871747b3843c51d3dec40088cad0d149fa62ae7d2c97297002474a55876" exitCode=6
Jul 17 10:13:19.217715 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:19.217242    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n" event={"ID":"fd0a31d2-f50a-41b0-8c1d-81f8d797f007","Type":"ContainerDied","Data":"72d13871747b3843c51d3dec40088cad0d149fa62ae7d2c97297002474a55876"}
Jul 17 10:13:19.217715 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:19.217319    2571 scope.go:117] "RemoveContainer" containerID="0c82f359546d036efd40189073e668e02c1917b2b4605dfed3cc297e01f69108"
Jul 17 10:13:19.217715 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:19.217586    2571 scope.go:117] "RemoveContainer" containerID="72d13871747b3843c51d3dec40088cad0d149fa62ae7d2c97297002474a55876"
Jul 17 10:13:19.217881 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:19.217841    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n_odh-ai-gateway-infra(fd0a31d2-f50a-41b0-8c1d-81f8d797f007)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007"
Jul 17 10:13:20.225168 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:20.225135    2571 scope.go:117] "RemoveContainer" containerID="72d13871747b3843c51d3dec40088cad0d149fa62ae7d2c97297002474a55876"
Jul 17 10:13:20.225592 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:20.225358    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n_odh-ai-gateway-infra(fd0a31d2-f50a-41b0-8c1d-81f8d797f007)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007"
Jul 17 10:13:23.825345 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:23.825306    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"]
Jul 17 10:13:23.825806 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:23.825790    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerName="revoke-keys"
Jul 17 10:13:23.825871 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:23.825808    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerName="revoke-keys"
Jul 17 10:13:23.825922 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:23.825911    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="89e8a616-d36f-46b3-a2d6-58bb3d02923d" containerName="revoke-keys"
Jul 17 10:13:23.830522 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:23.830502    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"
Jul 17 10:13:23.845161 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:23.845127    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"]
Jul 17 10:13:23.934082 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:23.934041    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416090e3-dff9-4e59-9299-fd47a9c36d32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r\" (UID: \"416090e3-dff9-4e59-9299-fd47a9c36d32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"
Jul 17 10:13:24.034719 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:24.034678    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416090e3-dff9-4e59-9299-fd47a9c36d32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r\" (UID: \"416090e3-dff9-4e59-9299-fd47a9c36d32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"
Jul 17 10:13:24.035359 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:24.035340    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416090e3-dff9-4e59-9299-fd47a9c36d32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r\" (UID: \"416090e3-dff9-4e59-9299-fd47a9c36d32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"
Jul 17 10:13:24.142348 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:24.142248    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"
Jul 17 10:13:24.307187 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:24.307144    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"]
Jul 17 10:13:24.312059 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:13:24.312029    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416090e3_dff9_4e59_9299_fd47a9c36d32.slice/crio-2396081cf05a067ee363af3ed6b9dfb9d941fc469ed9471d61f12bd1d26802b8 WatchSource:0}: Error finding container 2396081cf05a067ee363af3ed6b9dfb9d941fc469ed9471d61f12bd1d26802b8: Status 404 returned error can't find the container with id 2396081cf05a067ee363af3ed6b9dfb9d941fc469ed9471d61f12bd1d26802b8
Jul 17 10:13:25.266388 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:25.266342    2571 generic.go:356] "Generic (PLEG): container finished" podID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerID="ca86dd3796c9e52ce11fe597cdbbb773be3bd60e28e4cf6ebed92d0c35fcf234" exitCode=6
Jul 17 10:13:25.266803 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:25.266416    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r" event={"ID":"416090e3-dff9-4e59-9299-fd47a9c36d32","Type":"ContainerDied","Data":"ca86dd3796c9e52ce11fe597cdbbb773be3bd60e28e4cf6ebed92d0c35fcf234"}
Jul 17 10:13:25.266803 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:25.266515    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r" event={"ID":"416090e3-dff9-4e59-9299-fd47a9c36d32","Type":"ContainerStarted","Data":"2396081cf05a067ee363af3ed6b9dfb9d941fc469ed9471d61f12bd1d26802b8"}
Jul 17 10:13:25.266803 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:25.266644    2571 scope.go:117] "RemoveContainer" containerID="ca86dd3796c9e52ce11fe597cdbbb773be3bd60e28e4cf6ebed92d0c35fcf234"
Jul 17 10:13:26.281377 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:26.281343    2571 generic.go:356] "Generic (PLEG): container finished" podID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerID="a7c2dd70887c360079938e4754de7324bf1ad0107a1b7258b74c0088a207e292" exitCode=6
Jul 17 10:13:26.281812 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:26.281422    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r" event={"ID":"416090e3-dff9-4e59-9299-fd47a9c36d32","Type":"ContainerDied","Data":"a7c2dd70887c360079938e4754de7324bf1ad0107a1b7258b74c0088a207e292"}
Jul 17 10:13:26.281812 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:26.281537    2571 scope.go:117] "RemoveContainer" containerID="ca86dd3796c9e52ce11fe597cdbbb773be3bd60e28e4cf6ebed92d0c35fcf234"
Jul 17 10:13:26.281812 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:26.281688    2571 scope.go:117] "RemoveContainer" containerID="a7c2dd70887c360079938e4754de7324bf1ad0107a1b7258b74c0088a207e292"
Jul 17 10:13:26.281997 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:26.281908    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r_odh-ai-gateway-infra(416090e3-dff9-4e59-9299-fd47a9c36d32)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32"
Jul 17 10:13:27.289675 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:27.289647    2571 scope.go:117] "RemoveContainer" containerID="a7c2dd70887c360079938e4754de7324bf1ad0107a1b7258b74c0088a207e292"
Jul 17 10:13:27.290069 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:27.289867    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r_odh-ai-gateway-infra(416090e3-dff9-4e59-9299-fd47a9c36d32)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32"
Jul 17 10:13:33.752595 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:33.752560    2571 scope.go:117] "RemoveContainer" containerID="72d13871747b3843c51d3dec40088cad0d149fa62ae7d2c97297002474a55876"
Jul 17 10:13:34.331374 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:34.331293    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n" event={"ID":"fd0a31d2-f50a-41b0-8c1d-81f8d797f007","Type":"ContainerStarted","Data":"c99896f23f364054a7545bf1123181f3cfcb904fa3afea3f1a49bc0de78ec6f2"}
Jul 17 10:13:34.892004 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:34.891965    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"]
Jul 17 10:13:34.895735 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:34.895718    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"
Jul 17 10:13:34.908164 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:34.908135    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"]
Jul 17 10:13:34.934906 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:34.934871    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fb4588f-8eb1-4a41-b59e-2f9a422eb7db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4\" (UID: \"5fb4588f-8eb1-4a41-b59e-2f9a422eb7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"
Jul 17 10:13:35.035675 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:35.035640    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fb4588f-8eb1-4a41-b59e-2f9a422eb7db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4\" (UID: \"5fb4588f-8eb1-4a41-b59e-2f9a422eb7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"
Jul 17 10:13:35.036278 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:35.036258    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fb4588f-8eb1-4a41-b59e-2f9a422eb7db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4\" (UID: \"5fb4588f-8eb1-4a41-b59e-2f9a422eb7db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"
Jul 17 10:13:35.206905 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:35.206861    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"
Jul 17 10:13:35.342537 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:35.342508    2571 generic.go:356] "Generic (PLEG): container finished" podID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerID="c99896f23f364054a7545bf1123181f3cfcb904fa3afea3f1a49bc0de78ec6f2" exitCode=6
Jul 17 10:13:35.342714 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:35.342572    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n" event={"ID":"fd0a31d2-f50a-41b0-8c1d-81f8d797f007","Type":"ContainerDied","Data":"c99896f23f364054a7545bf1123181f3cfcb904fa3afea3f1a49bc0de78ec6f2"}
Jul 17 10:13:35.342714 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:35.342622    2571 scope.go:117] "RemoveContainer" containerID="72d13871747b3843c51d3dec40088cad0d149fa62ae7d2c97297002474a55876"
Jul 17 10:13:35.342973 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:35.342949    2571 scope.go:117] "RemoveContainer" containerID="c99896f23f364054a7545bf1123181f3cfcb904fa3afea3f1a49bc0de78ec6f2"
Jul 17 10:13:35.343257 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:35.343218    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n_odh-ai-gateway-infra(fd0a31d2-f50a-41b0-8c1d-81f8d797f007)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007"
Jul 17 10:13:35.353152 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:35.353120    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"]
Jul 17 10:13:35.353287 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:13:35.353265    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb4588f_8eb1_4a41_b59e_2f9a422eb7db.slice/crio-c1b5088dab4c474b6c38beb4b68a71d0967e174a3828ea97ec02ec2cf4de3518 WatchSource:0}: Error finding container c1b5088dab4c474b6c38beb4b68a71d0967e174a3828ea97ec02ec2cf4de3518: Status 404 returned error can't find the container with id c1b5088dab4c474b6c38beb4b68a71d0967e174a3828ea97ec02ec2cf4de3518
Jul 17 10:13:35.366938 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:35.366913    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"]
Jul 17 10:13:36.352690 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:36.352662    2571 generic.go:356] "Generic (PLEG): container finished" podID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerID="1ac811d2bb9f661b5a3edd3a94d7e277d4d18610a6133424ea2de7fd3db10c6a" exitCode=6
Jul 17 10:13:36.353039 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:36.352743    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4" event={"ID":"5fb4588f-8eb1-4a41-b59e-2f9a422eb7db","Type":"ContainerDied","Data":"1ac811d2bb9f661b5a3edd3a94d7e277d4d18610a6133424ea2de7fd3db10c6a"}
Jul 17 10:13:36.353039 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:36.352796    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4" event={"ID":"5fb4588f-8eb1-4a41-b59e-2f9a422eb7db","Type":"ContainerStarted","Data":"c1b5088dab4c474b6c38beb4b68a71d0967e174a3828ea97ec02ec2cf4de3518"}
Jul 17 10:13:36.353039 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:36.352962    2571 scope.go:117] "RemoveContainer" containerID="1ac811d2bb9f661b5a3edd3a94d7e277d4d18610a6133424ea2de7fd3db10c6a"
Jul 17 10:13:36.501176 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:36.501144    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"
Jul 17 10:13:36.547849 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:36.547822    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd0a31d2-f50a-41b0-8c1d-81f8d797f007-maas-api-service-ca\") pod \"fd0a31d2-f50a-41b0-8c1d-81f8d797f007\" (UID: \"fd0a31d2-f50a-41b0-8c1d-81f8d797f007\") "
Jul 17 10:13:36.548166 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:36.548144    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd0a31d2-f50a-41b0-8c1d-81f8d797f007-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fd0a31d2-f50a-41b0-8c1d-81f8d797f007" (UID: "fd0a31d2-f50a-41b0-8c1d-81f8d797f007"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:13:36.649212 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:36.649187    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd0a31d2-f50a-41b0-8c1d-81f8d797f007-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:13:37.360116 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.360081    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"
Jul 17 10:13:37.360587 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.360122    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n" event={"ID":"fd0a31d2-f50a-41b0-8c1d-81f8d797f007","Type":"ContainerDied","Data":"bdcd51d756e7e31c6b69c06a1c3419bf71cd017e447c185e0a425fc924747b10"}
Jul 17 10:13:37.360587 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.360231    2571 scope.go:117] "RemoveContainer" containerID="c99896f23f364054a7545bf1123181f3cfcb904fa3afea3f1a49bc0de78ec6f2"
Jul 17 10:13:37.362211 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.362184    2571 generic.go:356] "Generic (PLEG): container finished" podID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerID="c01cf2c9c9d04e3cab5e758d2be7bb5aec917dd809a89b46950db9734dfd8730" exitCode=6
Jul 17 10:13:37.362326 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.362270    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4" event={"ID":"5fb4588f-8eb1-4a41-b59e-2f9a422eb7db","Type":"ContainerDied","Data":"c01cf2c9c9d04e3cab5e758d2be7bb5aec917dd809a89b46950db9734dfd8730"}
Jul 17 10:13:37.362478 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.362441    2571 scope.go:117] "RemoveContainer" containerID="c01cf2c9c9d04e3cab5e758d2be7bb5aec917dd809a89b46950db9734dfd8730"
Jul 17 10:13:37.362668 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:37.362649    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4_odh-ai-gateway-infra(5fb4588f-8eb1-4a41-b59e-2f9a422eb7db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db"
Jul 17 10:13:37.370760 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.370742    2571 scope.go:117] "RemoveContainer" containerID="1ac811d2bb9f661b5a3edd3a94d7e277d4d18610a6133424ea2de7fd3db10c6a"
Jul 17 10:13:37.399656 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.399623    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"]
Jul 17 10:13:37.408302 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.408270    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-bp76n"]
Jul 17 10:13:37.671660 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.671572    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 10:13:37.683996 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.683972    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jhvwq_29ba670a-9f03-4942-be1e-79807bf25031/ovn-acl-logging/0.log"
Jul 17 10:13:37.760085 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:37.760053    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" path="/var/lib/kubelet/pods/fd0a31d2-f50a-41b0-8c1d-81f8d797f007/volumes"
Jul 17 10:13:38.370628 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:38.370541    2571 scope.go:117] "RemoveContainer" containerID="c01cf2c9c9d04e3cab5e758d2be7bb5aec917dd809a89b46950db9734dfd8730"
Jul 17 10:13:38.371058 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:38.370779    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4_odh-ai-gateway-infra(5fb4588f-8eb1-4a41-b59e-2f9a422eb7db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db"
Jul 17 10:13:40.752992 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:40.752955    2571 scope.go:117] "RemoveContainer" containerID="a7c2dd70887c360079938e4754de7324bf1ad0107a1b7258b74c0088a207e292"
Jul 17 10:13:41.389103 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:41.389065    2571 generic.go:356] "Generic (PLEG): container finished" podID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerID="8d3768b1e27b28ee3b5c57a1f6cb417deda2a7bf83a1d424c78c0231e390f1d4" exitCode=6
Jul 17 10:13:41.389313 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:41.389139    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r" event={"ID":"416090e3-dff9-4e59-9299-fd47a9c36d32","Type":"ContainerDied","Data":"8d3768b1e27b28ee3b5c57a1f6cb417deda2a7bf83a1d424c78c0231e390f1d4"}
Jul 17 10:13:41.389313 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:41.389198    2571 scope.go:117] "RemoveContainer" containerID="a7c2dd70887c360079938e4754de7324bf1ad0107a1b7258b74c0088a207e292"
Jul 17 10:13:41.389543 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:41.389515    2571 scope.go:117] "RemoveContainer" containerID="8d3768b1e27b28ee3b5c57a1f6cb417deda2a7bf83a1d424c78c0231e390f1d4"
Jul 17 10:13:41.389773 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:41.389752    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r_odh-ai-gateway-infra(416090e3-dff9-4e59-9299-fd47a9c36d32)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32"
Jul 17 10:13:42.429832 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:42.429797    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"]
Jul 17 10:13:42.572222 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:42.572195    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"
Jul 17 10:13:42.603754 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:42.603721    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416090e3-dff9-4e59-9299-fd47a9c36d32-maas-api-service-ca\") pod \"416090e3-dff9-4e59-9299-fd47a9c36d32\" (UID: \"416090e3-dff9-4e59-9299-fd47a9c36d32\") "
Jul 17 10:13:42.604180 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:42.604151    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/416090e3-dff9-4e59-9299-fd47a9c36d32-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "416090e3-dff9-4e59-9299-fd47a9c36d32" (UID: "416090e3-dff9-4e59-9299-fd47a9c36d32"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:13:42.704919 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:42.704882    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416090e3-dff9-4e59-9299-fd47a9c36d32-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:13:43.405928 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:43.405897    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"
Jul 17 10:13:43.406128 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:43.405898    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r" event={"ID":"416090e3-dff9-4e59-9299-fd47a9c36d32","Type":"ContainerDied","Data":"2396081cf05a067ee363af3ed6b9dfb9d941fc469ed9471d61f12bd1d26802b8"}
Jul 17 10:13:43.406128 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:43.406046    2571 scope.go:117] "RemoveContainer" containerID="8d3768b1e27b28ee3b5c57a1f6cb417deda2a7bf83a1d424c78c0231e390f1d4"
Jul 17 10:13:43.431388 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:43.431359    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"]
Jul 17 10:13:43.435300 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:43.435274    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-zrk6r"]
Jul 17 10:13:43.758804 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:43.758769    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32" path="/var/lib/kubelet/pods/416090e3-dff9-4e59-9299-fd47a9c36d32/volumes"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.987741    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"]
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.988800    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.988825    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.988852    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.988861    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.988874    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.988884    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.988927    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.988937    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.988945    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.988953    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.989139    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.989151    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.989164    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.989179    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerName="revoke-keys"
Jul 17 10:13:45.991274 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.989191    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerName="revoke-keys"
Jul 17 10:13:45.995182 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.995152    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"
Jul 17 10:13:45.999878 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:45.999806    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"]
Jul 17 10:13:46.039301 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:46.039267    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/732a299d-1e7c-4e86-851b-dc67c9af0098-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5\" (UID: \"732a299d-1e7c-4e86-851b-dc67c9af0098\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"
Jul 17 10:13:46.140082 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:46.140049    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/732a299d-1e7c-4e86-851b-dc67c9af0098-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5\" (UID: \"732a299d-1e7c-4e86-851b-dc67c9af0098\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"
Jul 17 10:13:46.140725 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:46.140702    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/732a299d-1e7c-4e86-851b-dc67c9af0098-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5\" (UID: \"732a299d-1e7c-4e86-851b-dc67c9af0098\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"
Jul 17 10:13:46.308440 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:46.308350    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"
Jul 17 10:13:46.476655 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:46.476625    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"]
Jul 17 10:13:46.477983 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:13:46.477957    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod732a299d_1e7c_4e86_851b_dc67c9af0098.slice/crio-9c8a381cafa25465cffca04969d6851e981f00f0d04c7ede0a644fda980f37da WatchSource:0}: Error finding container 9c8a381cafa25465cffca04969d6851e981f00f0d04c7ede0a644fda980f37da: Status 404 returned error can't find the container with id 9c8a381cafa25465cffca04969d6851e981f00f0d04c7ede0a644fda980f37da
Jul 17 10:13:47.433704 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:47.433623    2571 generic.go:356] "Generic (PLEG): container finished" podID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerID="d12a2da4f23d94668328cd9833193ac238af76bb9697bb9481d87556e9884c16" exitCode=6
Jul 17 10:13:47.433704 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:47.433667    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5" event={"ID":"732a299d-1e7c-4e86-851b-dc67c9af0098","Type":"ContainerDied","Data":"d12a2da4f23d94668328cd9833193ac238af76bb9697bb9481d87556e9884c16"}
Jul 17 10:13:47.434097 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:47.433744    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5" event={"ID":"732a299d-1e7c-4e86-851b-dc67c9af0098","Type":"ContainerStarted","Data":"9c8a381cafa25465cffca04969d6851e981f00f0d04c7ede0a644fda980f37da"}
Jul 17 10:13:47.434097 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:47.433982    2571 scope.go:117] "RemoveContainer" containerID="d12a2da4f23d94668328cd9833193ac238af76bb9697bb9481d87556e9884c16"
Jul 17 10:13:48.441336 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:48.441303    2571 generic.go:356] "Generic (PLEG): container finished" podID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerID="4c575cd25fdd84cb24487ef1eda395ce362a749e9176d9c2b1481f7c04e600b4" exitCode=6
Jul 17 10:13:48.441815 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:48.441385    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5" event={"ID":"732a299d-1e7c-4e86-851b-dc67c9af0098","Type":"ContainerDied","Data":"4c575cd25fdd84cb24487ef1eda395ce362a749e9176d9c2b1481f7c04e600b4"}
Jul 17 10:13:48.441815 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:48.441456    2571 scope.go:117] "RemoveContainer" containerID="d12a2da4f23d94668328cd9833193ac238af76bb9697bb9481d87556e9884c16"
Jul 17 10:13:48.441815 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:48.441635    2571 scope.go:117] "RemoveContainer" containerID="4c575cd25fdd84cb24487ef1eda395ce362a749e9176d9c2b1481f7c04e600b4"
Jul 17 10:13:48.441967 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:48.441908    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5_odh-ai-gateway-infra(732a299d-1e7c-4e86-851b-dc67c9af0098)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098"
Jul 17 10:13:49.449321 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:49.449295    2571 scope.go:117] "RemoveContainer" containerID="4c575cd25fdd84cb24487ef1eda395ce362a749e9176d9c2b1481f7c04e600b4"
Jul 17 10:13:49.449808 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:49.449536    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5_odh-ai-gateway-infra(732a299d-1e7c-4e86-851b-dc67c9af0098)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098"
Jul 17 10:13:52.752609 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:52.752578    2571 scope.go:117] "RemoveContainer" containerID="c01cf2c9c9d04e3cab5e758d2be7bb5aec917dd809a89b46950db9734dfd8730"
Jul 17 10:13:53.475024 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:53.474995    2571 generic.go:356] "Generic (PLEG): container finished" podID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerID="c21b7ec62062eed4253cf090fb3df458be2a5769b6742a1b5fb1097b781ea371" exitCode=6
Jul 17 10:13:53.475193 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:53.475071    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4" event={"ID":"5fb4588f-8eb1-4a41-b59e-2f9a422eb7db","Type":"ContainerDied","Data":"c21b7ec62062eed4253cf090fb3df458be2a5769b6742a1b5fb1097b781ea371"}
Jul 17 10:13:53.475193 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:53.475148    2571 scope.go:117] "RemoveContainer" containerID="c01cf2c9c9d04e3cab5e758d2be7bb5aec917dd809a89b46950db9734dfd8730"
Jul 17 10:13:53.475473 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:53.475441    2571 scope.go:117] "RemoveContainer" containerID="c21b7ec62062eed4253cf090fb3df458be2a5769b6742a1b5fb1097b781ea371"
Jul 17 10:13:53.475736 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:13:53.475708    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4_odh-ai-gateway-infra(5fb4588f-8eb1-4a41-b59e-2f9a422eb7db)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db"
Jul 17 10:13:54.508900 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:54.508860    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"]
Jul 17 10:13:54.653839 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:54.653814    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"
Jul 17 10:13:54.816256 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:54.816174    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fb4588f-8eb1-4a41-b59e-2f9a422eb7db-maas-api-service-ca\") pod \"5fb4588f-8eb1-4a41-b59e-2f9a422eb7db\" (UID: \"5fb4588f-8eb1-4a41-b59e-2f9a422eb7db\") "
Jul 17 10:13:54.816574 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:54.816551    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fb4588f-8eb1-4a41-b59e-2f9a422eb7db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" (UID: "5fb4588f-8eb1-4a41-b59e-2f9a422eb7db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:13:54.917130 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:54.917091    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fb4588f-8eb1-4a41-b59e-2f9a422eb7db-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:13:55.494432 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:55.494403    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"
Jul 17 10:13:55.494631 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:55.494451    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4" event={"ID":"5fb4588f-8eb1-4a41-b59e-2f9a422eb7db","Type":"ContainerDied","Data":"c1b5088dab4c474b6c38beb4b68a71d0967e174a3828ea97ec02ec2cf4de3518"}
Jul 17 10:13:55.494631 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:55.494562    2571 scope.go:117] "RemoveContainer" containerID="c21b7ec62062eed4253cf090fb3df458be2a5769b6742a1b5fb1097b781ea371"
Jul 17 10:13:55.520005 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:55.519975    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"]
Jul 17 10:13:55.528109 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:55.528080    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-d5lq4"]
Jul 17 10:13:55.759174 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:13:55.759097    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" path="/var/lib/kubelet/pods/5fb4588f-8eb1-4a41-b59e-2f9a422eb7db/volumes"
Jul 17 10:14:02.752531 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:02.752434    2571 scope.go:117] "RemoveContainer" containerID="4c575cd25fdd84cb24487ef1eda395ce362a749e9176d9c2b1481f7c04e600b4"
Jul 17 10:14:03.551415 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:03.551381    2571 generic.go:356] "Generic (PLEG): container finished" podID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerID="e2a8ec34fe4d237fc6090e62f60f82712034842809346d3b4c886b7f078c8e7d" exitCode=6
Jul 17 10:14:03.551612 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:03.551439    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5" event={"ID":"732a299d-1e7c-4e86-851b-dc67c9af0098","Type":"ContainerDied","Data":"e2a8ec34fe4d237fc6090e62f60f82712034842809346d3b4c886b7f078c8e7d"}
Jul 17 10:14:03.551612 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:03.551546    2571 scope.go:117] "RemoveContainer" containerID="4c575cd25fdd84cb24487ef1eda395ce362a749e9176d9c2b1481f7c04e600b4"
Jul 17 10:14:03.551842 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:03.551820    2571 scope.go:117] "RemoveContainer" containerID="e2a8ec34fe4d237fc6090e62f60f82712034842809346d3b4c886b7f078c8e7d"
Jul 17 10:14:03.552070 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:03.552049    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5_odh-ai-gateway-infra(732a299d-1e7c-4e86-851b-dc67c9af0098)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098"
Jul 17 10:14:04.587795 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:04.587751    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"]
Jul 17 10:14:04.731812 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:04.731785    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"
Jul 17 10:14:04.914862 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:04.914776    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/732a299d-1e7c-4e86-851b-dc67c9af0098-maas-api-service-ca\") pod \"732a299d-1e7c-4e86-851b-dc67c9af0098\" (UID: \"732a299d-1e7c-4e86-851b-dc67c9af0098\") "
Jul 17 10:14:04.915160 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:04.915133    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/732a299d-1e7c-4e86-851b-dc67c9af0098-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "732a299d-1e7c-4e86-851b-dc67c9af0098" (UID: "732a299d-1e7c-4e86-851b-dc67c9af0098"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:05.016094 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:05.016058    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/732a299d-1e7c-4e86-851b-dc67c9af0098-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:14:05.568613 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:05.568572    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"
Jul 17 10:14:05.568613 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:05.568587    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5" event={"ID":"732a299d-1e7c-4e86-851b-dc67c9af0098","Type":"ContainerDied","Data":"9c8a381cafa25465cffca04969d6851e981f00f0d04c7ede0a644fda980f37da"}
Jul 17 10:14:05.568865 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:05.568653    2571 scope.go:117] "RemoveContainer" containerID="e2a8ec34fe4d237fc6090e62f60f82712034842809346d3b4c886b7f078c8e7d"
Jul 17 10:14:05.594943 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:05.594915    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"]
Jul 17 10:14:05.602754 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:05.602732    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-xt8p5"]
Jul 17 10:14:05.758773 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:05.758738    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098" path="/var/lib/kubelet/pods/732a299d-1e7c-4e86-851b-dc67c9af0098/volumes"
Jul 17 10:14:08.802159 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:08.802127    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"]
Jul 17 10:14:13.859117 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859083    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"]
Jul 17 10:14:13.859583 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859543    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerName="revoke-keys"
Jul 17 10:14:13.859583 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859556    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerName="revoke-keys"
Jul 17 10:14:13.859583 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859568    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerName="revoke-keys"
Jul 17 10:14:13.859583 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859574    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="416090e3-dff9-4e59-9299-fd47a9c36d32" containerName="revoke-keys"
Jul 17 10:14:13.859583 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859581    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerName="revoke-keys"
Jul 17 10:14:13.859583 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859586    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859595    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859601    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859612    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859617    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859623    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859628    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859635    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859640    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859700    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859706    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859713    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859720    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="732a299d-1e7c-4e86-851b-dc67c9af0098" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859727    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd0a31d2-f50a-41b0-8c1d-81f8d797f007" containerName="revoke-keys"
Jul 17 10:14:13.859793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.859734    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerName="revoke-keys"
Jul 17 10:14:13.864175 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.864158    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"
Jul 17 10:14:13.867755 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.867737    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:14:13.877441 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.877416    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"]
Jul 17 10:14:13.888168 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.888133    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/514176f0-5dc1-4ee0-9355-a81a6a2658fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl\" (UID: \"514176f0-5dc1-4ee0-9355-a81a6a2658fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"
Jul 17 10:14:13.988952 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.988912    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/514176f0-5dc1-4ee0-9355-a81a6a2658fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl\" (UID: \"514176f0-5dc1-4ee0-9355-a81a6a2658fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"
Jul 17 10:14:13.989562 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:13.989540    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/514176f0-5dc1-4ee0-9355-a81a6a2658fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl\" (UID: \"514176f0-5dc1-4ee0-9355-a81a6a2658fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"
Jul 17 10:14:14.175032 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:14.174944    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"
Jul 17 10:14:14.321621 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:14.321593    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"]
Jul 17 10:14:14.322986 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:14:14.322959    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514176f0_5dc1_4ee0_9355_a81a6a2658fb.slice/crio-c35e6aa0bf4c592ad770d301990ce70a1a0f0b72f618c60cb6479b56cb98c0da WatchSource:0}: Error finding container c35e6aa0bf4c592ad770d301990ce70a1a0f0b72f618c60cb6479b56cb98c0da: Status 404 returned error can't find the container with id c35e6aa0bf4c592ad770d301990ce70a1a0f0b72f618c60cb6479b56cb98c0da
Jul 17 10:14:14.624986 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:14.624939    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl" event={"ID":"514176f0-5dc1-4ee0-9355-a81a6a2658fb","Type":"ContainerStarted","Data":"c35e6aa0bf4c592ad770d301990ce70a1a0f0b72f618c60cb6479b56cb98c0da"}
Jul 17 10:14:15.633368 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:15.633329    2571 generic.go:356] "Generic (PLEG): container finished" podID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerID="9b88b23db8253fbd108efc399a54df9ec006611841a388da686f086289c62de0" exitCode=6
Jul 17 10:14:15.633789 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:15.633409    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl" event={"ID":"514176f0-5dc1-4ee0-9355-a81a6a2658fb","Type":"ContainerDied","Data":"9b88b23db8253fbd108efc399a54df9ec006611841a388da686f086289c62de0"}
Jul 17 10:14:15.633789 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:15.633599    2571 scope.go:117] "RemoveContainer" containerID="9b88b23db8253fbd108efc399a54df9ec006611841a388da686f086289c62de0"
Jul 17 10:14:16.363790 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.363698    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"]
Jul 17 10:14:16.364233 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.364220    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fb4588f-8eb1-4a41-b59e-2f9a422eb7db" containerName="revoke-keys"
Jul 17 10:14:16.367407 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.367390    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"
Jul 17 10:14:16.379717 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.379690    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"]
Jul 17 10:14:16.409338 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.409300    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2ad8b4c-53bf-49f3-b36b-5166164e573f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs\" (UID: \"a2ad8b4c-53bf-49f3-b36b-5166164e573f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"
Jul 17 10:14:16.510155 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.510121    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2ad8b4c-53bf-49f3-b36b-5166164e573f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs\" (UID: \"a2ad8b4c-53bf-49f3-b36b-5166164e573f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"
Jul 17 10:14:16.510780 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.510761    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2ad8b4c-53bf-49f3-b36b-5166164e573f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs\" (UID: \"a2ad8b4c-53bf-49f3-b36b-5166164e573f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"
Jul 17 10:14:16.642528 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.642426    2571 generic.go:356] "Generic (PLEG): container finished" podID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerID="07115fd64c352b5821d2b89b20021aaee638e07d213ca2be4a24304395197293" exitCode=6
Jul 17 10:14:16.642528 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.642503    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl" event={"ID":"514176f0-5dc1-4ee0-9355-a81a6a2658fb","Type":"ContainerDied","Data":"07115fd64c352b5821d2b89b20021aaee638e07d213ca2be4a24304395197293"}
Jul 17 10:14:16.642994 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.642561    2571 scope.go:117] "RemoveContainer" containerID="9b88b23db8253fbd108efc399a54df9ec006611841a388da686f086289c62de0"
Jul 17 10:14:16.642994 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.642756    2571 scope.go:117] "RemoveContainer" containerID="07115fd64c352b5821d2b89b20021aaee638e07d213ca2be4a24304395197293"
Jul 17 10:14:16.643062 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:16.643007    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl_odh-ai-gateway-infra(514176f0-5dc1-4ee0-9355-a81a6a2658fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb"
Jul 17 10:14:16.678363 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:16.678332    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"
Jul 17 10:14:17.056291 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:17.056265    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"]
Jul 17 10:14:17.058106 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:14:17.058078    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ad8b4c_53bf_49f3_b36b_5166164e573f.slice/crio-25cf812f04fb007b314703bb85715a644538c76baa94892a0af8b502b8acdf69 WatchSource:0}: Error finding container 25cf812f04fb007b314703bb85715a644538c76baa94892a0af8b502b8acdf69: Status 404 returned error can't find the container with id 25cf812f04fb007b314703bb85715a644538c76baa94892a0af8b502b8acdf69
Jul 17 10:14:17.650677 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:17.650634    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs" event={"ID":"a2ad8b4c-53bf-49f3-b36b-5166164e573f","Type":"ContainerStarted","Data":"c14aad4fdd05d9f85cdc2832eb534f07bd04be9cafa3361bfd7fa5c40b2e0a25"}
Jul 17 10:14:17.651065 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:17.650709    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs" event={"ID":"a2ad8b4c-53bf-49f3-b36b-5166164e573f","Type":"ContainerStarted","Data":"25cf812f04fb007b314703bb85715a644538c76baa94892a0af8b502b8acdf69"}
Jul 17 10:14:17.652594 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:17.652577    2571 scope.go:117] "RemoveContainer" containerID="07115fd64c352b5821d2b89b20021aaee638e07d213ca2be4a24304395197293"
Jul 17 10:14:17.652828 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:17.652801    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl_odh-ai-gateway-infra(514176f0-5dc1-4ee0-9355-a81a6a2658fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb"
Jul 17 10:14:17.692177 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:17.692120    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs" podStartSLOduration=1.692103632 podStartE2EDuration="1.692103632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:14:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:14:17.691786971 +0000 UTC m=+3341.756607690" watchObservedRunningTime="2026-07-17 10:14:17.692103632 +0000 UTC m=+3341.756924353"
Jul 17 10:14:18.661007 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:18.660974    2571 generic.go:356] "Generic (PLEG): container finished" podID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerID="c14aad4fdd05d9f85cdc2832eb534f07bd04be9cafa3361bfd7fa5c40b2e0a25" exitCode=6
Jul 17 10:14:18.661481 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:18.661086    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs" event={"ID":"a2ad8b4c-53bf-49f3-b36b-5166164e573f","Type":"ContainerDied","Data":"c14aad4fdd05d9f85cdc2832eb534f07bd04be9cafa3361bfd7fa5c40b2e0a25"}
Jul 17 10:14:18.661481 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:18.661296    2571 scope.go:117] "RemoveContainer" containerID="c14aad4fdd05d9f85cdc2832eb534f07bd04be9cafa3361bfd7fa5c40b2e0a25"
Jul 17 10:14:19.669104 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:19.669013    2571 generic.go:356] "Generic (PLEG): container finished" podID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerID="f1aec1660d992304dbab8deceb35e43c4fee10dd9883ad5362aded6792082e69" exitCode=6
Jul 17 10:14:19.669104 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:19.669056    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs" event={"ID":"a2ad8b4c-53bf-49f3-b36b-5166164e573f","Type":"ContainerDied","Data":"f1aec1660d992304dbab8deceb35e43c4fee10dd9883ad5362aded6792082e69"}
Jul 17 10:14:19.669614 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:19.669131    2571 scope.go:117] "RemoveContainer" containerID="c14aad4fdd05d9f85cdc2832eb534f07bd04be9cafa3361bfd7fa5c40b2e0a25"
Jul 17 10:14:19.669614 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:19.669283    2571 scope.go:117] "RemoveContainer" containerID="f1aec1660d992304dbab8deceb35e43c4fee10dd9883ad5362aded6792082e69"
Jul 17 10:14:19.669614 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:19.669540    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs_odh-ai-gateway-infra(a2ad8b4c-53bf-49f3-b36b-5166164e573f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f"
Jul 17 10:14:20.677421 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:20.677390    2571 scope.go:117] "RemoveContainer" containerID="f1aec1660d992304dbab8deceb35e43c4fee10dd9883ad5362aded6792082e69"
Jul 17 10:14:20.677846 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:20.677623    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs_odh-ai-gateway-infra(a2ad8b4c-53bf-49f3-b36b-5166164e573f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f"
Jul 17 10:14:23.821986 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:23.821942    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m" podUID="e7e8ec60-9975-4d24-890a-2ba0a6dcc114" containerName="main" containerID="cri-o://b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95" gracePeriod=45
Jul 17 10:14:24.083633 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.083577    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:14:24.184402 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.184370    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-home\") pod \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") "
Jul 17 10:14:24.184402 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.184406    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mzl4\" (UniqueName: \"kubernetes.io/projected/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-kube-api-access-7mzl4\") pod \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") "
Jul 17 10:14:24.184657 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.184449    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-dshm\") pod \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") "
Jul 17 10:14:24.184657 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.184506    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tls-certs\") pod \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") "
Jul 17 10:14:24.184657 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.184537    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tmp-dir\") pod \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") "
Jul 17 10:14:24.184657 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.184566    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-model-cache\") pod \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\" (UID: \"e7e8ec60-9975-4d24-890a-2ba0a6dcc114\") "
Jul 17 10:14:24.184860 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.184669    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-home" (OuterVolumeSpecName: "home") pod "e7e8ec60-9975-4d24-890a-2ba0a6dcc114" (UID: "e7e8ec60-9975-4d24-890a-2ba0a6dcc114"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:24.184918 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.184831    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-home\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:14:24.184973 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.184942    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e7e8ec60-9975-4d24-890a-2ba0a6dcc114" (UID: "e7e8ec60-9975-4d24-890a-2ba0a6dcc114"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:24.184973 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.184958    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-model-cache" (OuterVolumeSpecName: "model-cache") pod "e7e8ec60-9975-4d24-890a-2ba0a6dcc114" (UID: "e7e8ec60-9975-4d24-890a-2ba0a6dcc114"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:24.186696 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.186672    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-kube-api-access-7mzl4" (OuterVolumeSpecName: "kube-api-access-7mzl4") pod "e7e8ec60-9975-4d24-890a-2ba0a6dcc114" (UID: "e7e8ec60-9975-4d24-890a-2ba0a6dcc114"). InnerVolumeSpecName "kube-api-access-7mzl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:14:24.186796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.186706    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e7e8ec60-9975-4d24-890a-2ba0a6dcc114" (UID: "e7e8ec60-9975-4d24-890a-2ba0a6dcc114"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 10:14:24.186796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.186728    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-dshm" (OuterVolumeSpecName: "dshm") pod "e7e8ec60-9975-4d24-890a-2ba0a6dcc114" (UID: "e7e8ec60-9975-4d24-890a-2ba0a6dcc114"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:24.286029 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.285989    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tmp-dir\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:14:24.286029 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.286022    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-model-cache\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:14:24.286029 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.286033    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mzl4\" (UniqueName: \"kubernetes.io/projected/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-kube-api-access-7mzl4\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:14:24.286260 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.286043    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-dshm\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:14:24.286260 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.286052    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7e8ec60-9975-4d24-890a-2ba0a6dcc114-tls-certs\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:14:24.703185 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.703152    2571 generic.go:356] "Generic (PLEG): container finished" podID="e7e8ec60-9975-4d24-890a-2ba0a6dcc114" containerID="b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95" exitCode=0
Jul 17 10:14:24.703371 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.703202    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"
Jul 17 10:14:24.703371 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.703235    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m" event={"ID":"e7e8ec60-9975-4d24-890a-2ba0a6dcc114","Type":"ContainerDied","Data":"b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95"}
Jul 17 10:14:24.703371 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.703279    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m" event={"ID":"e7e8ec60-9975-4d24-890a-2ba0a6dcc114","Type":"ContainerDied","Data":"58ad706ead42277fb8c9941d0b3d78acb2beb394204b000657aba68b22ca9c13"}
Jul 17 10:14:24.703371 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.703304    2571 scope.go:117] "RemoveContainer" containerID="b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95"
Jul 17 10:14:24.713390 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.713372    2571 scope.go:117] "RemoveContainer" containerID="b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95"
Jul 17 10:14:24.713681 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:24.713660    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95\": container with ID starting with b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95 not found: ID does not exist" containerID="b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95"
Jul 17 10:14:24.713775 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.713687    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95"} err="failed to get container status \"b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95\": rpc error: code = NotFound desc = could not find container \"b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95\": container with ID starting with b2a902e08a63bb59898329910c250243cc05744569068b97f41db5d7077c4c95 not found: ID does not exist"
Jul 17 10:14:24.728836 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.728808    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"]
Jul 17 10:14:24.736487 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:24.736416    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8ae1eb/sub-test-model-8ae1eb-kserve-77fd9c87c9-jtt8m"]
Jul 17 10:14:25.759434 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:25.759397    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7e8ec60-9975-4d24-890a-2ba0a6dcc114" path="/var/lib/kubelet/pods/e7e8ec60-9975-4d24-890a-2ba0a6dcc114/volumes"
Jul 17 10:14:31.753016 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:31.752977    2571 scope.go:117] "RemoveContainer" containerID="f1aec1660d992304dbab8deceb35e43c4fee10dd9883ad5362aded6792082e69"
Jul 17 10:14:32.752507 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:32.752443    2571 scope.go:117] "RemoveContainer" containerID="07115fd64c352b5821d2b89b20021aaee638e07d213ca2be4a24304395197293"
Jul 17 10:14:32.766872 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:32.766845    2571 generic.go:356] "Generic (PLEG): container finished" podID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerID="d907667b0487a705a475a02fc5d647e1b1be4f9fd13d8470e30551f4c74f3f4c" exitCode=6
Jul 17 10:14:32.767231 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:32.766918    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs" event={"ID":"a2ad8b4c-53bf-49f3-b36b-5166164e573f","Type":"ContainerDied","Data":"d907667b0487a705a475a02fc5d647e1b1be4f9fd13d8470e30551f4c74f3f4c"}
Jul 17 10:14:32.767231 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:32.766974    2571 scope.go:117] "RemoveContainer" containerID="f1aec1660d992304dbab8deceb35e43c4fee10dd9883ad5362aded6792082e69"
Jul 17 10:14:32.767337 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:32.767249    2571 scope.go:117] "RemoveContainer" containerID="d907667b0487a705a475a02fc5d647e1b1be4f9fd13d8470e30551f4c74f3f4c"
Jul 17 10:14:32.767435 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:32.767415    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs_odh-ai-gateway-infra(a2ad8b4c-53bf-49f3-b36b-5166164e573f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f"
Jul 17 10:14:33.775156 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:33.775119    2571 generic.go:356] "Generic (PLEG): container finished" podID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerID="80c417f4754b9808fd378c393d6ed1c7e28cfcf8240c6330c4175738350c0a89" exitCode=6
Jul 17 10:14:33.775626 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:33.775188    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl" event={"ID":"514176f0-5dc1-4ee0-9355-a81a6a2658fb","Type":"ContainerDied","Data":"80c417f4754b9808fd378c393d6ed1c7e28cfcf8240c6330c4175738350c0a89"}
Jul 17 10:14:33.775626 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:33.775249    2571 scope.go:117] "RemoveContainer" containerID="07115fd64c352b5821d2b89b20021aaee638e07d213ca2be4a24304395197293"
Jul 17 10:14:33.775626 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:33.775584    2571 scope.go:117] "RemoveContainer" containerID="80c417f4754b9808fd378c393d6ed1c7e28cfcf8240c6330c4175738350c0a89"
Jul 17 10:14:33.775848 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:33.775822    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl_odh-ai-gateway-infra(514176f0-5dc1-4ee0-9355-a81a6a2658fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb"
Jul 17 10:14:33.814020 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:33.813985    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"]
Jul 17 10:14:33.959731 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:33.959705    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"
Jul 17 10:14:34.074273 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.074184    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2ad8b4c-53bf-49f3-b36b-5166164e573f-maas-api-service-ca\") pod \"a2ad8b4c-53bf-49f3-b36b-5166164e573f\" (UID: \"a2ad8b4c-53bf-49f3-b36b-5166164e573f\") "
Jul 17 10:14:34.074609 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.074587    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2ad8b4c-53bf-49f3-b36b-5166164e573f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a2ad8b4c-53bf-49f3-b36b-5166164e573f" (UID: "a2ad8b4c-53bf-49f3-b36b-5166164e573f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:34.175095 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.175054    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2ad8b4c-53bf-49f3-b36b-5166164e573f-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:14:34.784380 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.784350    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"
Jul 17 10:14:34.784956 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.784398    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs" event={"ID":"a2ad8b4c-53bf-49f3-b36b-5166164e573f","Type":"ContainerDied","Data":"25cf812f04fb007b314703bb85715a644538c76baa94892a0af8b502b8acdf69"}
Jul 17 10:14:34.784956 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.784488    2571 scope.go:117] "RemoveContainer" containerID="d907667b0487a705a475a02fc5d647e1b1be4f9fd13d8470e30551f4c74f3f4c"
Jul 17 10:14:34.815436 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.815407    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"]
Jul 17 10:14:34.828056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.828029    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"]
Jul 17 10:14:34.858096 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.858059    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-fsnjs"]
Jul 17 10:14:34.954810 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.954781    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"]
Jul 17 10:14:34.955447 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.955430    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerName="revoke-keys"
Jul 17 10:14:34.955504 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.955451    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerName="revoke-keys"
Jul 17 10:14:34.955504 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.955484    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7e8ec60-9975-4d24-890a-2ba0a6dcc114" containerName="main"
Jul 17 10:14:34.955504 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.955494    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e8ec60-9975-4d24-890a-2ba0a6dcc114" containerName="main"
Jul 17 10:14:34.955595 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.955524    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerName="revoke-keys"
Jul 17 10:14:34.955595 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.955534    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerName="revoke-keys"
Jul 17 10:14:34.955656 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.955650    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7e8ec60-9975-4d24-890a-2ba0a6dcc114" containerName="main"
Jul 17 10:14:34.955688 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.955663    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerName="revoke-keys"
Jul 17 10:14:34.955688 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.955674    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerName="revoke-keys"
Jul 17 10:14:34.955688 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.955684    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerName="revoke-keys"
Jul 17 10:14:34.960602 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.960582    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"
Jul 17 10:14:34.968235 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.968208    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"]
Jul 17 10:14:34.971643 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:34.971618    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"
Jul 17 10:14:35.084173 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.084083    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/514176f0-5dc1-4ee0-9355-a81a6a2658fb-maas-api-service-ca\") pod \"514176f0-5dc1-4ee0-9355-a81a6a2658fb\" (UID: \"514176f0-5dc1-4ee0-9355-a81a6a2658fb\") "
Jul 17 10:14:35.084326 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.084256    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/561e32bb-34f5-4fd1-b082-52fabeaa0ea2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46\" (UID: \"561e32bb-34f5-4fd1-b082-52fabeaa0ea2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"
Jul 17 10:14:35.084499 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.084445    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/514176f0-5dc1-4ee0-9355-a81a6a2658fb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "514176f0-5dc1-4ee0-9355-a81a6a2658fb" (UID: "514176f0-5dc1-4ee0-9355-a81a6a2658fb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:35.185259 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.185223    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/561e32bb-34f5-4fd1-b082-52fabeaa0ea2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46\" (UID: \"561e32bb-34f5-4fd1-b082-52fabeaa0ea2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"
Jul 17 10:14:35.185432 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.185283    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/514176f0-5dc1-4ee0-9355-a81a6a2658fb-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:14:35.185908 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.185888    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/561e32bb-34f5-4fd1-b082-52fabeaa0ea2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46\" (UID: \"561e32bb-34f5-4fd1-b082-52fabeaa0ea2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"
Jul 17 10:14:35.277380 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.277342    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"
Jul 17 10:14:35.430451 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.430357    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"]
Jul 17 10:14:35.433209 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:14:35.433176    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561e32bb_34f5_4fd1_b082_52fabeaa0ea2.slice/crio-56fb689f12898aee5c9e330fd0eaa8b65cc8a684e96b89faf44eeba22cac2c8a WatchSource:0}: Error finding container 56fb689f12898aee5c9e330fd0eaa8b65cc8a684e96b89faf44eeba22cac2c8a: Status 404 returned error can't find the container with id 56fb689f12898aee5c9e330fd0eaa8b65cc8a684e96b89faf44eeba22cac2c8a
Jul 17 10:14:35.766338 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.766303    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" path="/var/lib/kubelet/pods/a2ad8b4c-53bf-49f3-b36b-5166164e573f/volumes"
Jul 17 10:14:35.793989 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.793958    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"
Jul 17 10:14:35.793989 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.793966    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl" event={"ID":"514176f0-5dc1-4ee0-9355-a81a6a2658fb","Type":"ContainerDied","Data":"c35e6aa0bf4c592ad770d301990ce70a1a0f0b72f618c60cb6479b56cb98c0da"}
Jul 17 10:14:35.794454 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.794027    2571 scope.go:117] "RemoveContainer" containerID="80c417f4754b9808fd378c393d6ed1c7e28cfcf8240c6330c4175738350c0a89"
Jul 17 10:14:35.795861 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.795830    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46" event={"ID":"561e32bb-34f5-4fd1-b082-52fabeaa0ea2","Type":"ContainerStarted","Data":"1620a5c4d6e551bfdd227d17a1a028755934ec6f6316c66494e266fc3973a3e4"}
Jul 17 10:14:35.796001 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.795981    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46" event={"ID":"561e32bb-34f5-4fd1-b082-52fabeaa0ea2","Type":"ContainerStarted","Data":"56fb689f12898aee5c9e330fd0eaa8b65cc8a684e96b89faf44eeba22cac2c8a"}
Jul 17 10:14:35.821084 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.821058    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"]
Jul 17 10:14:35.830442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.830420    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-mwhdl"]
Jul 17 10:14:35.869357 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:35.869310    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46" podStartSLOduration=1.869291673 podStartE2EDuration="1.869291673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:14:35.863729406 +0000 UTC m=+3359.928550139" watchObservedRunningTime="2026-07-17 10:14:35.869291673 +0000 UTC m=+3359.934112396"
Jul 17 10:14:36.057935 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.057896    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"]
Jul 17 10:14:36.058490 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.058455    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerName="revoke-keys"
Jul 17 10:14:36.058490 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.058487    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ad8b4c-53bf-49f3-b36b-5166164e573f" containerName="revoke-keys"
Jul 17 10:14:36.058634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.058505    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerName="revoke-keys"
Jul 17 10:14:36.058634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.058512    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerName="revoke-keys"
Jul 17 10:14:36.058634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.058539    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerName="revoke-keys"
Jul 17 10:14:36.058634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.058545    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerName="revoke-keys"
Jul 17 10:14:36.058634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.058613    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerName="revoke-keys"
Jul 17 10:14:36.058634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.058624    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerName="revoke-keys"
Jul 17 10:14:36.058634 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.058630    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerName="revoke-keys"
Jul 17 10:14:36.063113 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.063096    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"
Jul 17 10:14:36.085695 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.085655    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"]
Jul 17 10:14:36.195526 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.195480    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd901c0-581b-44dd-bf76-cd6f5dae6ced-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49\" (UID: \"1cd901c0-581b-44dd-bf76-cd6f5dae6ced\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"
Jul 17 10:14:36.296954 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.296908    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd901c0-581b-44dd-bf76-cd6f5dae6ced-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49\" (UID: \"1cd901c0-581b-44dd-bf76-cd6f5dae6ced\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"
Jul 17 10:14:36.297647 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.297628    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd901c0-581b-44dd-bf76-cd6f5dae6ced-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49\" (UID: \"1cd901c0-581b-44dd-bf76-cd6f5dae6ced\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"
Jul 17 10:14:36.374135 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.374048    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"
Jul 17 10:14:36.521635 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.521609    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"]
Jul 17 10:14:36.522274 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:14:36.522254    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd901c0_581b_44dd_bf76_cd6f5dae6ced.slice/crio-f832fdb6ece0acf84e00253a89bc81d9bc816bc010e39afeaf05f54875665b9e WatchSource:0}: Error finding container f832fdb6ece0acf84e00253a89bc81d9bc816bc010e39afeaf05f54875665b9e: Status 404 returned error can't find the container with id f832fdb6ece0acf84e00253a89bc81d9bc816bc010e39afeaf05f54875665b9e
Jul 17 10:14:36.804905 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.804867    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49" event={"ID":"1cd901c0-581b-44dd-bf76-cd6f5dae6ced","Type":"ContainerStarted","Data":"f832fdb6ece0acf84e00253a89bc81d9bc816bc010e39afeaf05f54875665b9e"}
Jul 17 10:14:36.807222 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.807195    2571 generic.go:356] "Generic (PLEG): container finished" podID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerID="1620a5c4d6e551bfdd227d17a1a028755934ec6f6316c66494e266fc3973a3e4" exitCode=6
Jul 17 10:14:36.807358 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.807240    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46" event={"ID":"561e32bb-34f5-4fd1-b082-52fabeaa0ea2","Type":"ContainerDied","Data":"1620a5c4d6e551bfdd227d17a1a028755934ec6f6316c66494e266fc3973a3e4"}
Jul 17 10:14:36.807557 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:36.807541    2571 scope.go:117] "RemoveContainer" containerID="1620a5c4d6e551bfdd227d17a1a028755934ec6f6316c66494e266fc3973a3e4"
Jul 17 10:14:37.760240 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:37.760211    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" path="/var/lib/kubelet/pods/514176f0-5dc1-4ee0-9355-a81a6a2658fb/volumes"
Jul 17 10:14:37.814917 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:37.814885    2571 generic.go:356] "Generic (PLEG): container finished" podID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerID="5abf482b15c491f730cf01ccc691739bb31983fcff3c37dd884473be33a9e40e" exitCode=6
Jul 17 10:14:37.815337 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:37.815031    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46" event={"ID":"561e32bb-34f5-4fd1-b082-52fabeaa0ea2","Type":"ContainerDied","Data":"5abf482b15c491f730cf01ccc691739bb31983fcff3c37dd884473be33a9e40e"}
Jul 17 10:14:37.815337 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:37.815092    2571 scope.go:117] "RemoveContainer" containerID="1620a5c4d6e551bfdd227d17a1a028755934ec6f6316c66494e266fc3973a3e4"
Jul 17 10:14:37.815337 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:37.815243    2571 scope.go:117] "RemoveContainer" containerID="5abf482b15c491f730cf01ccc691739bb31983fcff3c37dd884473be33a9e40e"
Jul 17 10:14:37.815496 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:37.815447    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46_odh-ai-gateway-infra(561e32bb-34f5-4fd1-b082-52fabeaa0ea2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2"
Jul 17 10:14:37.816795 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:37.816775    2571 generic.go:356] "Generic (PLEG): container finished" podID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerID="74b666cf0ea0947fea597f4ded34aac7772e26d34a300d196a11f3fa149302ec" exitCode=6
Jul 17 10:14:37.816897 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:37.816802    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49" event={"ID":"1cd901c0-581b-44dd-bf76-cd6f5dae6ced","Type":"ContainerDied","Data":"74b666cf0ea0947fea597f4ded34aac7772e26d34a300d196a11f3fa149302ec"}
Jul 17 10:14:37.816990 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:37.816974    2571 scope.go:117] "RemoveContainer" containerID="74b666cf0ea0947fea597f4ded34aac7772e26d34a300d196a11f3fa149302ec"
Jul 17 10:14:38.824056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:38.824022    2571 generic.go:356] "Generic (PLEG): container finished" podID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerID="495d42d686cee66a763ab4d184b4398824614496781d589fb94e2f5a87e52557" exitCode=6
Jul 17 10:14:38.824514 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:38.824105    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49" event={"ID":"1cd901c0-581b-44dd-bf76-cd6f5dae6ced","Type":"ContainerDied","Data":"495d42d686cee66a763ab4d184b4398824614496781d589fb94e2f5a87e52557"}
Jul 17 10:14:38.824514 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:38.824164    2571 scope.go:117] "RemoveContainer" containerID="74b666cf0ea0947fea597f4ded34aac7772e26d34a300d196a11f3fa149302ec"
Jul 17 10:14:38.824514 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:38.824298    2571 scope.go:117] "RemoveContainer" containerID="495d42d686cee66a763ab4d184b4398824614496781d589fb94e2f5a87e52557"
Jul 17 10:14:38.824695 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:38.824538    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49_odh-ai-gateway-infra(1cd901c0-581b-44dd-bf76-cd6f5dae6ced)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced"
Jul 17 10:14:38.826097 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:38.826079    2571 scope.go:117] "RemoveContainer" containerID="5abf482b15c491f730cf01ccc691739bb31983fcff3c37dd884473be33a9e40e"
Jul 17 10:14:38.826240 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:38.826221    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46_odh-ai-gateway-infra(561e32bb-34f5-4fd1-b082-52fabeaa0ea2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2"
Jul 17 10:14:39.835025 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:39.834999    2571 scope.go:117] "RemoveContainer" containerID="495d42d686cee66a763ab4d184b4398824614496781d589fb94e2f5a87e52557"
Jul 17 10:14:39.835509 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:39.835187    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49_odh-ai-gateway-infra(1cd901c0-581b-44dd-bf76-cd6f5dae6ced)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced"
Jul 17 10:14:52.752925 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:52.752894    2571 scope.go:117] "RemoveContainer" containerID="495d42d686cee66a763ab4d184b4398824614496781d589fb94e2f5a87e52557"
Jul 17 10:14:52.753390 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:52.752967    2571 scope.go:117] "RemoveContainer" containerID="5abf482b15c491f730cf01ccc691739bb31983fcff3c37dd884473be33a9e40e"
Jul 17 10:14:53.920152 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:53.920122    2571 generic.go:356] "Generic (PLEG): container finished" podID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerID="6ebb5f7842f51fec79a644bc1c6b09baceb5289496832da5344063414f9ce6c2" exitCode=6
Jul 17 10:14:53.920651 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:53.920201    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49" event={"ID":"1cd901c0-581b-44dd-bf76-cd6f5dae6ced","Type":"ContainerDied","Data":"6ebb5f7842f51fec79a644bc1c6b09baceb5289496832da5344063414f9ce6c2"}
Jul 17 10:14:53.920651 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:53.920270    2571 scope.go:117] "RemoveContainer" containerID="495d42d686cee66a763ab4d184b4398824614496781d589fb94e2f5a87e52557"
Jul 17 10:14:53.920651 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:53.920632    2571 scope.go:117] "RemoveContainer" containerID="6ebb5f7842f51fec79a644bc1c6b09baceb5289496832da5344063414f9ce6c2"
Jul 17 10:14:53.920929 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:53.920902    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49_odh-ai-gateway-infra(1cd901c0-581b-44dd-bf76-cd6f5dae6ced)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced"
Jul 17 10:14:53.922208 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:53.922188    2571 generic.go:356] "Generic (PLEG): container finished" podID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerID="fdc56b734b16c436fbdbf50c7ed4360eed395beeadc8a9d2f8eeddc37f00673b" exitCode=6
Jul 17 10:14:53.922315 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:53.922263    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46" event={"ID":"561e32bb-34f5-4fd1-b082-52fabeaa0ea2","Type":"ContainerDied","Data":"fdc56b734b16c436fbdbf50c7ed4360eed395beeadc8a9d2f8eeddc37f00673b"}
Jul 17 10:14:53.922572 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:53.922558    2571 scope.go:117] "RemoveContainer" containerID="fdc56b734b16c436fbdbf50c7ed4360eed395beeadc8a9d2f8eeddc37f00673b"
Jul 17 10:14:53.922766 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:14:53.922748    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46_odh-ai-gateway-infra(561e32bb-34f5-4fd1-b082-52fabeaa0ea2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2"
Jul 17 10:14:53.932201 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:53.932185    2571 scope.go:117] "RemoveContainer" containerID="5abf482b15c491f730cf01ccc691739bb31983fcff3c37dd884473be33a9e40e"
Jul 17 10:14:54.950611 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:54.950565    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"]
Jul 17 10:14:54.971092 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:54.971059    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"]
Jul 17 10:14:55.124777 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.124750    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"
Jul 17 10:14:55.166526 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.166494    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"
Jul 17 10:14:55.290532 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.290405    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd901c0-581b-44dd-bf76-cd6f5dae6ced-maas-api-service-ca\") pod \"1cd901c0-581b-44dd-bf76-cd6f5dae6ced\" (UID: \"1cd901c0-581b-44dd-bf76-cd6f5dae6ced\") "
Jul 17 10:14:55.290532 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.290456    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/561e32bb-34f5-4fd1-b082-52fabeaa0ea2-maas-api-service-ca\") pod \"561e32bb-34f5-4fd1-b082-52fabeaa0ea2\" (UID: \"561e32bb-34f5-4fd1-b082-52fabeaa0ea2\") "
Jul 17 10:14:55.290789 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.290761    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cd901c0-581b-44dd-bf76-cd6f5dae6ced-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1cd901c0-581b-44dd-bf76-cd6f5dae6ced" (UID: "1cd901c0-581b-44dd-bf76-cd6f5dae6ced"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:55.290877 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.290858    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/561e32bb-34f5-4fd1-b082-52fabeaa0ea2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "561e32bb-34f5-4fd1-b082-52fabeaa0ea2" (UID: "561e32bb-34f5-4fd1-b082-52fabeaa0ea2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:55.391555 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.391504    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cd901c0-581b-44dd-bf76-cd6f5dae6ced-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:14:55.391555 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.391546    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/561e32bb-34f5-4fd1-b082-52fabeaa0ea2-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:14:55.939215 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.939184    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"
Jul 17 10:14:55.939410 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.939185    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49" event={"ID":"1cd901c0-581b-44dd-bf76-cd6f5dae6ced","Type":"ContainerDied","Data":"f832fdb6ece0acf84e00253a89bc81d9bc816bc010e39afeaf05f54875665b9e"}
Jul 17 10:14:55.939410 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.939351    2571 scope.go:117] "RemoveContainer" containerID="6ebb5f7842f51fec79a644bc1c6b09baceb5289496832da5344063414f9ce6c2"
Jul 17 10:14:55.940755 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.940736    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"
Jul 17 10:14:55.940887 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.940761    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46" event={"ID":"561e32bb-34f5-4fd1-b082-52fabeaa0ea2","Type":"ContainerDied","Data":"56fb689f12898aee5c9e330fd0eaa8b65cc8a684e96b89faf44eeba22cac2c8a"}
Jul 17 10:14:55.949349 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.949330    2571 scope.go:117] "RemoveContainer" containerID="fdc56b734b16c436fbdbf50c7ed4360eed395beeadc8a9d2f8eeddc37f00673b"
Jul 17 10:14:55.959022 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.959001    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"]
Jul 17 10:14:55.963949 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.963929    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-67k49"]
Jul 17 10:14:55.976705 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.976680    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"]
Jul 17 10:14:55.982932 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:55.982911    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-sxk46"]
Jul 17 10:14:57.759753 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:57.759720    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" path="/var/lib/kubelet/pods/1cd901c0-581b-44dd-bf76-cd6f5dae6ced/volumes"
Jul 17 10:14:57.760134 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:14:57.760081    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" path="/var/lib/kubelet/pods/561e32bb-34f5-4fd1-b082-52fabeaa0ea2/volumes"
Jul 17 10:15:00.156919 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.156882    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45"]
Jul 17 10:15:00.157334 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157320    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerName="revoke-keys"
Jul 17 10:15:00.157334 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157335    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerName="revoke-keys"
Jul 17 10:15:00.157409 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157353    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerName="revoke-keys"
Jul 17 10:15:00.157409 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157359    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerName="revoke-keys"
Jul 17 10:15:00.157409 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157365    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerName="revoke-keys"
Jul 17 10:15:00.157409 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157372    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerName="revoke-keys"
Jul 17 10:15:00.157409 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157379    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerName="revoke-keys"
Jul 17 10:15:00.157409 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157384    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerName="revoke-keys"
Jul 17 10:15:00.157409 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157394    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerName="revoke-keys"
Jul 17 10:15:00.157409 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157399    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerName="revoke-keys"
Jul 17 10:15:00.157668 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157411    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerName="revoke-keys"
Jul 17 10:15:00.157668 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157417    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="514176f0-5dc1-4ee0-9355-a81a6a2658fb" containerName="revoke-keys"
Jul 17 10:15:00.157668 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157498    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerName="revoke-keys"
Jul 17 10:15:00.157668 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157506    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerName="revoke-keys"
Jul 17 10:15:00.157668 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157513    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerName="revoke-keys"
Jul 17 10:15:00.157668 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157519    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="561e32bb-34f5-4fd1-b082-52fabeaa0ea2" containerName="revoke-keys"
Jul 17 10:15:00.157668 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.157524    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerName="revoke-keys"
Jul 17 10:15:00.162172 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.162147    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45"
Jul 17 10:15:00.162485 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.162455    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl"]
Jul 17 10:15:00.162897 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.162882    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerName="revoke-keys"
Jul 17 10:15:00.162965 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.162899    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerName="revoke-keys"
Jul 17 10:15:00.163015 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.162973    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cd901c0-581b-44dd-bf76-cd6f5dae6ced" containerName="revoke-keys"
Jul 17 10:15:00.165939 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.165919    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn"]
Jul 17 10:15:00.166081 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.166059    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl"
Jul 17 10:15:00.166317 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.166300    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:15:00.169198 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.169181    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn"
Jul 17 10:15:00.175374 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.175206    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45"
Jul 17 10:15:00.175569 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.175552    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45"]
Jul 17 10:15:00.180893 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.180871    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl"]
Jul 17 10:15:00.185238 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.185214    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn"]
Jul 17 10:15:00.190528 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.190510    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl"
Jul 17 10:15:00.234984 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.234959    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn"
Jul 17 10:15:00.352185 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.352151    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45"]
Jul 17 10:15:00.356913 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:15:00.356707    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a72cc1a_6f3e_4ea5_83d9_b8e18555d5f9.slice/crio-cc69d72f25a3211fd20ae8eca5c9c9dd79d1fa7d40203698c2283c5c7bbc5b42 WatchSource:0}: Error finding container cc69d72f25a3211fd20ae8eca5c9c9dd79d1fa7d40203698c2283c5c7bbc5b42: Status 404 returned error can't find the container with id cc69d72f25a3211fd20ae8eca5c9c9dd79d1fa7d40203698c2283c5c7bbc5b42
Jul 17 10:15:00.416420 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.416394    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn"]
Jul 17 10:15:00.419121 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:15:00.419084    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa687682_dbf6_4234_aa37_895de3ebf353.slice/crio-afd084065ef8b81bc46ea2045f5042e4be2192495a3153adc874e96829ab4e7e WatchSource:0}: Error finding container afd084065ef8b81bc46ea2045f5042e4be2192495a3153adc874e96829ab4e7e: Status 404 returned error can't find the container with id afd084065ef8b81bc46ea2045f5042e4be2192495a3153adc874e96829ab4e7e
Jul 17 10:15:00.585473 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.585434    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl"]
Jul 17 10:15:00.586163 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:15:00.586137    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9479e35_51f9_42f8_a00f_1d857d5de9bb.slice/crio-1c41644c80be41faaa6f2de4b46f7db67486a13a9da905011f1c259259de547d WatchSource:0}: Error finding container 1c41644c80be41faaa6f2de4b46f7db67486a13a9da905011f1c259259de547d: Status 404 returned error can't find the container with id 1c41644c80be41faaa6f2de4b46f7db67486a13a9da905011f1c259259de547d
Jul 17 10:15:00.973692 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.973652    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl" event={"ID":"c9479e35-51f9-42f8-a00f-1d857d5de9bb","Type":"ContainerStarted","Data":"c68c1f458932afa274c9731718ef820dcedcfd7dd824e73c1b13dd9544603e05"}
Jul 17 10:15:00.973901 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.973715    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl" event={"ID":"c9479e35-51f9-42f8-a00f-1d857d5de9bb","Type":"ContainerStarted","Data":"1c41644c80be41faaa6f2de4b46f7db67486a13a9da905011f1c259259de547d"}
Jul 17 10:15:00.975331 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.975297    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn" event={"ID":"fa687682-dbf6-4234-aa37-895de3ebf353","Type":"ContainerStarted","Data":"07cf3fe868b0f7b85f08a64b1dd57890e49b76f9f823d2cec8bf3e273c66babb"}
Jul 17 10:15:00.975487 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.975346    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn" event={"ID":"fa687682-dbf6-4234-aa37-895de3ebf353","Type":"ContainerStarted","Data":"afd084065ef8b81bc46ea2045f5042e4be2192495a3153adc874e96829ab4e7e"}
Jul 17 10:15:00.976835 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.976809    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45" event={"ID":"8a72cc1a-6f3e-4ea5-83d9-b8e18555d5f9","Type":"ContainerStarted","Data":"b0659877c09a79d4fb7d526ff6fb8dc295cb3fda3cea69170dfbe2eeeba61f60"}
Jul 17 10:15:00.976931 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:00.976850    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45" event={"ID":"8a72cc1a-6f3e-4ea5-83d9-b8e18555d5f9","Type":"ContainerStarted","Data":"cc69d72f25a3211fd20ae8eca5c9c9dd79d1fa7d40203698c2283c5c7bbc5b42"}
Jul 17 10:15:01.021203 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:01.021106    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn" podStartSLOduration=1.021091447 podStartE2EDuration="1.021091447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:01.018981499 +0000 UTC m=+3385.083802221" watchObservedRunningTime="2026-07-17 10:15:01.021091447 +0000 UTC m=+3385.085912169"
Jul 17 10:15:01.022931 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:01.022896    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl" podStartSLOduration=1.022885207 podStartE2EDuration="1.022885207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:00.995263905 +0000 UTC m=+3385.060084628" watchObservedRunningTime="2026-07-17 10:15:01.022885207 +0000 UTC m=+3385.087705929"
Jul 17 10:15:01.044821 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:01.044663    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45" podStartSLOduration=1.044644937 podStartE2EDuration="1.044644937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:01.043928842 +0000 UTC m=+3385.108749564" watchObservedRunningTime="2026-07-17 10:15:01.044644937 +0000 UTC m=+3385.109465662"
Jul 17 10:15:01.984267 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:01.984232    2571 generic.go:356] "Generic (PLEG): container finished" podID="fa687682-dbf6-4234-aa37-895de3ebf353" containerID="07cf3fe868b0f7b85f08a64b1dd57890e49b76f9f823d2cec8bf3e273c66babb" exitCode=0
Jul 17 10:15:01.984719 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:01.984315    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn" event={"ID":"fa687682-dbf6-4234-aa37-895de3ebf353","Type":"ContainerDied","Data":"07cf3fe868b0f7b85f08a64b1dd57890e49b76f9f823d2cec8bf3e273c66babb"}
Jul 17 10:15:01.985689 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:01.985669    2571 generic.go:356] "Generic (PLEG): container finished" podID="8a72cc1a-6f3e-4ea5-83d9-b8e18555d5f9" containerID="b0659877c09a79d4fb7d526ff6fb8dc295cb3fda3cea69170dfbe2eeeba61f60" exitCode=0
Jul 17 10:15:01.985841 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:01.985750    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45" event={"ID":"8a72cc1a-6f3e-4ea5-83d9-b8e18555d5f9","Type":"ContainerDied","Data":"b0659877c09a79d4fb7d526ff6fb8dc295cb3fda3cea69170dfbe2eeeba61f60"}
Jul 17 10:15:01.987193 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:01.987177    2571 generic.go:356] "Generic (PLEG): container finished" podID="c9479e35-51f9-42f8-a00f-1d857d5de9bb" containerID="c68c1f458932afa274c9731718ef820dcedcfd7dd824e73c1b13dd9544603e05" exitCode=0
Jul 17 10:15:01.987260 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:01.987203    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl" event={"ID":"c9479e35-51f9-42f8-a00f-1d857d5de9bb","Type":"ContainerDied","Data":"c68c1f458932afa274c9731718ef820dcedcfd7dd824e73c1b13dd9544603e05"}
Jul 17 10:15:03.154517 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:03.154485    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn"
Jul 17 10:15:03.201276 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:03.201251    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45"
Jul 17 10:15:03.214077 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:03.214056    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl"
Jul 17 10:15:03.254010 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:03.253920    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r"]
Jul 17 10:15:03.256251 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:03.256224    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738010-qq69r"]
Jul 17 10:15:03.759052 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:03.759017    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec724ac3-4239-4e4a-aa85-c063bd13c0c4" path="/var/lib/kubelet/pods/ec724ac3-4239-4e4a-aa85-c063bd13c0c4/volumes"
Jul 17 10:15:04.001615 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:04.001586    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl"
Jul 17 10:15:04.001800 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:04.001643    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl" event={"ID":"c9479e35-51f9-42f8-a00f-1d857d5de9bb","Type":"ContainerDied","Data":"1c41644c80be41faaa6f2de4b46f7db67486a13a9da905011f1c259259de547d"}
Jul 17 10:15:04.001800 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:04.001702    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c41644c80be41faaa6f2de4b46f7db67486a13a9da905011f1c259259de547d"
Jul 17 10:15:04.003124 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:04.003083    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn"
Jul 17 10:15:04.003124 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:04.003096    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738055-52gvn" event={"ID":"fa687682-dbf6-4234-aa37-895de3ebf353","Type":"ContainerDied","Data":"afd084065ef8b81bc46ea2045f5042e4be2192495a3153adc874e96829ab4e7e"}
Jul 17 10:15:04.003302 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:04.003143    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="afd084065ef8b81bc46ea2045f5042e4be2192495a3153adc874e96829ab4e7e"
Jul 17 10:15:04.004570 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:04.004550    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45" event={"ID":"8a72cc1a-6f3e-4ea5-83d9-b8e18555d5f9","Type":"ContainerDied","Data":"cc69d72f25a3211fd20ae8eca5c9c9dd79d1fa7d40203698c2283c5c7bbc5b42"}
Jul 17 10:15:04.004665 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:04.004583    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc69d72f25a3211fd20ae8eca5c9c9dd79d1fa7d40203698c2283c5c7bbc5b42"
Jul 17 10:15:04.004665 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:04.004598    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45"
Jul 17 10:15:06.431398 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.431372    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"]
Jul 17 10:15:06.431865 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.431850    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa687682-dbf6-4234-aa37-895de3ebf353" containerName="cleanup"
Jul 17 10:15:06.431865 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.431866    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa687682-dbf6-4234-aa37-895de3ebf353" containerName="cleanup"
Jul 17 10:15:06.431959 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.431888    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a72cc1a-6f3e-4ea5-83d9-b8e18555d5f9" containerName="cleanup"
Jul 17 10:15:06.431959 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.431893    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a72cc1a-6f3e-4ea5-83d9-b8e18555d5f9" containerName="cleanup"
Jul 17 10:15:06.431959 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.431907    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9479e35-51f9-42f8-a00f-1d857d5de9bb" containerName="cleanup"
Jul 17 10:15:06.431959 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.431913    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9479e35-51f9-42f8-a00f-1d857d5de9bb" containerName="cleanup"
Jul 17 10:15:06.432078 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.431970    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a72cc1a-6f3e-4ea5-83d9-b8e18555d5f9" containerName="cleanup"
Jul 17 10:15:06.432078 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.431980    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9479e35-51f9-42f8-a00f-1d857d5de9bb" containerName="cleanup"
Jul 17 10:15:06.432078 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.431989    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa687682-dbf6-4234-aa37-895de3ebf353" containerName="cleanup"
Jul 17 10:15:06.435294 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.435275    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"
Jul 17 10:15:06.442407 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.442380    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:15:06.454303 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.454278    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"]
Jul 17 10:15:06.487995 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.487966    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26518bae-0854-40a0-9cf9-8c6377e623a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7\" (UID: \"26518bae-0854-40a0-9cf9-8c6377e623a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"
Jul 17 10:15:06.589120 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.589079    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26518bae-0854-40a0-9cf9-8c6377e623a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7\" (UID: \"26518bae-0854-40a0-9cf9-8c6377e623a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"
Jul 17 10:15:06.589752 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.589731    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26518bae-0854-40a0-9cf9-8c6377e623a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7\" (UID: \"26518bae-0854-40a0-9cf9-8c6377e623a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"
Jul 17 10:15:06.746610 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.746575    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"
Jul 17 10:15:06.912198 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:06.912171    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"]
Jul 17 10:15:06.913441 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:15:06.913410    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26518bae_0854_40a0_9cf9_8c6377e623a2.slice/crio-01950e755cb1e3b8e5c386ac7aaa9227920b439d8ba23cabc11df5eaedeceeed WatchSource:0}: Error finding container 01950e755cb1e3b8e5c386ac7aaa9227920b439d8ba23cabc11df5eaedeceeed: Status 404 returned error can't find the container with id 01950e755cb1e3b8e5c386ac7aaa9227920b439d8ba23cabc11df5eaedeceeed
Jul 17 10:15:07.023102 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:07.023030    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7" event={"ID":"26518bae-0854-40a0-9cf9-8c6377e623a2","Type":"ContainerStarted","Data":"01950e755cb1e3b8e5c386ac7aaa9227920b439d8ba23cabc11df5eaedeceeed"}
Jul 17 10:15:08.030245 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:08.030205    2571 generic.go:356] "Generic (PLEG): container finished" podID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerID="ff6e82bdb330ef9573b779c5c0191eee087c5dd43667843cc93d2c2ac7d63e3a" exitCode=6
Jul 17 10:15:08.030726 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:08.030253    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7" event={"ID":"26518bae-0854-40a0-9cf9-8c6377e623a2","Type":"ContainerDied","Data":"ff6e82bdb330ef9573b779c5c0191eee087c5dd43667843cc93d2c2ac7d63e3a"}
Jul 17 10:15:08.030726 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:08.030530    2571 scope.go:117] "RemoveContainer" containerID="ff6e82bdb330ef9573b779c5c0191eee087c5dd43667843cc93d2c2ac7d63e3a"
Jul 17 10:15:09.037979 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:09.037885    2571 generic.go:356] "Generic (PLEG): container finished" podID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerID="e5137deffeb15bd6baaf202758c84e1bfc1379fe97bf469fdf243e35f2c37091" exitCode=6
Jul 17 10:15:09.037979 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:09.037962    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7" event={"ID":"26518bae-0854-40a0-9cf9-8c6377e623a2","Type":"ContainerDied","Data":"e5137deffeb15bd6baaf202758c84e1bfc1379fe97bf469fdf243e35f2c37091"}
Jul 17 10:15:09.038414 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:09.038020    2571 scope.go:117] "RemoveContainer" containerID="ff6e82bdb330ef9573b779c5c0191eee087c5dd43667843cc93d2c2ac7d63e3a"
Jul 17 10:15:09.038414 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:09.038149    2571 scope.go:117] "RemoveContainer" containerID="e5137deffeb15bd6baaf202758c84e1bfc1379fe97bf469fdf243e35f2c37091"
Jul 17 10:15:09.038414 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:09.038399    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7_odh-ai-gateway-infra(26518bae-0854-40a0-9cf9-8c6377e623a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2"
Jul 17 10:15:10.045796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:10.045766    2571 scope.go:117] "RemoveContainer" containerID="e5137deffeb15bd6baaf202758c84e1bfc1379fe97bf469fdf243e35f2c37091"
Jul 17 10:15:10.046147 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:10.045958    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7_odh-ai-gateway-infra(26518bae-0854-40a0-9cf9-8c6377e623a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2"
Jul 17 10:15:13.903967 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:13.903933    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"]
Jul 17 10:15:13.908982 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:13.908964    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"
Jul 17 10:15:13.916146 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:13.916120    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"]
Jul 17 10:15:13.958522 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:13.958493    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e47ec946-8125-4f2c-8170-05af5f43dd87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72\" (UID: \"e47ec946-8125-4f2c-8170-05af5f43dd87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"
Jul 17 10:15:14.059907 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:14.059870    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e47ec946-8125-4f2c-8170-05af5f43dd87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72\" (UID: \"e47ec946-8125-4f2c-8170-05af5f43dd87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"
Jul 17 10:15:14.060515 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:14.060497    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e47ec946-8125-4f2c-8170-05af5f43dd87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72\" (UID: \"e47ec946-8125-4f2c-8170-05af5f43dd87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"
Jul 17 10:15:14.222023 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:14.221992    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"
Jul 17 10:15:14.365541 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:14.365506    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"]
Jul 17 10:15:14.367439 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:15:14.367409    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode47ec946_8125_4f2c_8170_05af5f43dd87.slice/crio-8612fef9b08942d2e3c71a288747fd14bb504b80db3dc09427851be5fbe245a9 WatchSource:0}: Error finding container 8612fef9b08942d2e3c71a288747fd14bb504b80db3dc09427851be5fbe245a9: Status 404 returned error can't find the container with id 8612fef9b08942d2e3c71a288747fd14bb504b80db3dc09427851be5fbe245a9
Jul 17 10:15:15.078026 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:15.077995    2571 generic.go:356] "Generic (PLEG): container finished" podID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerID="27a4d782eed0c4ff1123f0a7cc1904c34a88356f5ba4a2aff9701f607d3c7111" exitCode=6
Jul 17 10:15:15.078385 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:15.078076    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72" event={"ID":"e47ec946-8125-4f2c-8170-05af5f43dd87","Type":"ContainerDied","Data":"27a4d782eed0c4ff1123f0a7cc1904c34a88356f5ba4a2aff9701f607d3c7111"}
Jul 17 10:15:15.078385 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:15.078124    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72" event={"ID":"e47ec946-8125-4f2c-8170-05af5f43dd87","Type":"ContainerStarted","Data":"8612fef9b08942d2e3c71a288747fd14bb504b80db3dc09427851be5fbe245a9"}
Jul 17 10:15:15.078385 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:15.078219    2571 scope.go:117] "RemoveContainer" containerID="27a4d782eed0c4ff1123f0a7cc1904c34a88356f5ba4a2aff9701f607d3c7111"
Jul 17 10:15:16.087203 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:16.087121    2571 generic.go:356] "Generic (PLEG): container finished" podID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerID="dd5cd8121137c64e253d026316bc0f6ead2954b96f326dbe9b436ee8d49415de" exitCode=6
Jul 17 10:15:16.089607 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:16.087209    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72" event={"ID":"e47ec946-8125-4f2c-8170-05af5f43dd87","Type":"ContainerDied","Data":"dd5cd8121137c64e253d026316bc0f6ead2954b96f326dbe9b436ee8d49415de"}
Jul 17 10:15:16.089607 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:16.087282    2571 scope.go:117] "RemoveContainer" containerID="27a4d782eed0c4ff1123f0a7cc1904c34a88356f5ba4a2aff9701f607d3c7111"
Jul 17 10:15:16.089607 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:16.087637    2571 scope.go:117] "RemoveContainer" containerID="dd5cd8121137c64e253d026316bc0f6ead2954b96f326dbe9b436ee8d49415de"
Jul 17 10:15:16.089607 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:16.088388    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72_odh-ai-gateway-infra(e47ec946-8125-4f2c-8170-05af5f43dd87)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87"
Jul 17 10:15:17.096147 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:17.096121    2571 scope.go:117] "RemoveContainer" containerID="dd5cd8121137c64e253d026316bc0f6ead2954b96f326dbe9b436ee8d49415de"
Jul 17 10:15:17.096544 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:17.096317    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72_odh-ai-gateway-infra(e47ec946-8125-4f2c-8170-05af5f43dd87)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87"
Jul 17 10:15:21.752152 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:21.752113    2571 scope.go:117] "RemoveContainer" containerID="e5137deffeb15bd6baaf202758c84e1bfc1379fe97bf469fdf243e35f2c37091"
Jul 17 10:15:22.127559 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:22.127524    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7" event={"ID":"26518bae-0854-40a0-9cf9-8c6377e623a2","Type":"ContainerStarted","Data":"071c3011c13ea06207dcc8957e2ed10ad7bf941fe9d91ec84120ce529c41cda7"}
Jul 17 10:15:22.150657 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:22.150602    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7" podStartSLOduration=16.150587634 podStartE2EDuration="16.150587634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:22.150199996 +0000 UTC m=+3406.215020719" watchObservedRunningTime="2026-07-17 10:15:22.150587634 +0000 UTC m=+3406.215408357"
Jul 17 10:15:23.135275 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:23.135234    2571 generic.go:356] "Generic (PLEG): container finished" podID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerID="071c3011c13ea06207dcc8957e2ed10ad7bf941fe9d91ec84120ce529c41cda7" exitCode=6
Jul 17 10:15:23.135762 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:23.135310    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7" event={"ID":"26518bae-0854-40a0-9cf9-8c6377e623a2","Type":"ContainerDied","Data":"071c3011c13ea06207dcc8957e2ed10ad7bf941fe9d91ec84120ce529c41cda7"}
Jul 17 10:15:23.135762 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:23.135432    2571 scope.go:117] "RemoveContainer" containerID="e5137deffeb15bd6baaf202758c84e1bfc1379fe97bf469fdf243e35f2c37091"
Jul 17 10:15:23.135864 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:23.135764    2571 scope.go:117] "RemoveContainer" containerID="071c3011c13ea06207dcc8957e2ed10ad7bf941fe9d91ec84120ce529c41cda7"
Jul 17 10:15:23.136013 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:23.135986    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7_odh-ai-gateway-infra(26518bae-0854-40a0-9cf9-8c6377e623a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2"
Jul 17 10:15:23.175169 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:23.175140    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"]
Jul 17 10:15:24.287898 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:24.287868    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"
Jul 17 10:15:24.348827 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:24.348791    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26518bae-0854-40a0-9cf9-8c6377e623a2-maas-api-service-ca\") pod \"26518bae-0854-40a0-9cf9-8c6377e623a2\" (UID: \"26518bae-0854-40a0-9cf9-8c6377e623a2\") "
Jul 17 10:15:24.349158 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:24.349136    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26518bae-0854-40a0-9cf9-8c6377e623a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "26518bae-0854-40a0-9cf9-8c6377e623a2" (UID: "26518bae-0854-40a0-9cf9-8c6377e623a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:24.450064 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:24.450030    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26518bae-0854-40a0-9cf9-8c6377e623a2-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:15:25.154899 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:25.154863    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"
Jul 17 10:15:25.154899 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:25.154873    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7" event={"ID":"26518bae-0854-40a0-9cf9-8c6377e623a2","Type":"ContainerDied","Data":"01950e755cb1e3b8e5c386ac7aaa9227920b439d8ba23cabc11df5eaedeceeed"}
Jul 17 10:15:25.155139 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:25.154962    2571 scope.go:117] "RemoveContainer" containerID="071c3011c13ea06207dcc8957e2ed10ad7bf941fe9d91ec84120ce529c41cda7"
Jul 17 10:15:25.185038 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:25.185009    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"]
Jul 17 10:15:25.190794 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:25.190771    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-65tl7"]
Jul 17 10:15:25.759040 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:25.759010    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2" path="/var/lib/kubelet/pods/26518bae-0854-40a0-9cf9-8c6377e623a2/volumes"
Jul 17 10:15:26.120774 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.120686    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"]
Jul 17 10:15:26.121148 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.121134    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerName="revoke-keys"
Jul 17 10:15:26.121191 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.121150    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerName="revoke-keys"
Jul 17 10:15:26.121191 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.121172    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerName="revoke-keys"
Jul 17 10:15:26.121191 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.121177    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerName="revoke-keys"
Jul 17 10:15:26.121191 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.121184    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerName="revoke-keys"
Jul 17 10:15:26.121191 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.121189    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerName="revoke-keys"
Jul 17 10:15:26.121355 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.121258    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerName="revoke-keys"
Jul 17 10:15:26.121355 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.121267    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerName="revoke-keys"
Jul 17 10:15:26.126008 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.125985    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"
Jul 17 10:15:26.135708 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.135681    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"]
Jul 17 10:15:26.165206 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.165177    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5aca8b8-b6e2-4072-93db-6498e7f85203-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh\" (UID: \"f5aca8b8-b6e2-4072-93db-6498e7f85203\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"
Jul 17 10:15:26.266360 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.266322    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5aca8b8-b6e2-4072-93db-6498e7f85203-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh\" (UID: \"f5aca8b8-b6e2-4072-93db-6498e7f85203\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"
Jul 17 10:15:26.266996 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.266974    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5aca8b8-b6e2-4072-93db-6498e7f85203-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh\" (UID: \"f5aca8b8-b6e2-4072-93db-6498e7f85203\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"
Jul 17 10:15:26.438551 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.438475    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"
Jul 17 10:15:26.579542 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:26.579516    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"]
Jul 17 10:15:26.581003 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:15:26.580974    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5aca8b8_b6e2_4072_93db_6498e7f85203.slice/crio-6fab43dffe8b76807f1d3f4c38b736fa77060b5d944963ba76e69c5b69aaf64a WatchSource:0}: Error finding container 6fab43dffe8b76807f1d3f4c38b736fa77060b5d944963ba76e69c5b69aaf64a: Status 404 returned error can't find the container with id 6fab43dffe8b76807f1d3f4c38b736fa77060b5d944963ba76e69c5b69aaf64a
Jul 17 10:15:27.169696 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:27.169665    2571 generic.go:356] "Generic (PLEG): container finished" podID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerID="db5489ea8b07487a880ac73786ca629215d7f93c1995e4d6b2ec67f811d87288" exitCode=6
Jul 17 10:15:27.170053 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:27.169750    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh" event={"ID":"f5aca8b8-b6e2-4072-93db-6498e7f85203","Type":"ContainerDied","Data":"db5489ea8b07487a880ac73786ca629215d7f93c1995e4d6b2ec67f811d87288"}
Jul 17 10:15:27.170053 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:27.169803    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh" event={"ID":"f5aca8b8-b6e2-4072-93db-6498e7f85203","Type":"ContainerStarted","Data":"6fab43dffe8b76807f1d3f4c38b736fa77060b5d944963ba76e69c5b69aaf64a"}
Jul 17 10:15:27.170053 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:27.169918    2571 scope.go:117] "RemoveContainer" containerID="db5489ea8b07487a880ac73786ca629215d7f93c1995e4d6b2ec67f811d87288"
Jul 17 10:15:28.178307 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:28.178181    2571 generic.go:356] "Generic (PLEG): container finished" podID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerID="dcc9f76b550a5f7f6f023a787dbdd4442929e81dc9da78f0514701d3d7a2547e" exitCode=6
Jul 17 10:15:28.178307 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:28.178265    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh" event={"ID":"f5aca8b8-b6e2-4072-93db-6498e7f85203","Type":"ContainerDied","Data":"dcc9f76b550a5f7f6f023a787dbdd4442929e81dc9da78f0514701d3d7a2547e"}
Jul 17 10:15:28.178771 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:28.178325    2571 scope.go:117] "RemoveContainer" containerID="db5489ea8b07487a880ac73786ca629215d7f93c1995e4d6b2ec67f811d87288"
Jul 17 10:15:28.178771 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:28.178422    2571 scope.go:117] "RemoveContainer" containerID="dcc9f76b550a5f7f6f023a787dbdd4442929e81dc9da78f0514701d3d7a2547e"
Jul 17 10:15:28.178771 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:28.178699    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh_odh-ai-gateway-infra(f5aca8b8-b6e2-4072-93db-6498e7f85203)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203"
Jul 17 10:15:29.186545 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:29.186518    2571 scope.go:117] "RemoveContainer" containerID="dcc9f76b550a5f7f6f023a787dbdd4442929e81dc9da78f0514701d3d7a2547e"
Jul 17 10:15:29.186896 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:29.186721    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh_odh-ai-gateway-infra(f5aca8b8-b6e2-4072-93db-6498e7f85203)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203"
Jul 17 10:15:30.752716 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:30.752686    2571 scope.go:117] "RemoveContainer" containerID="dd5cd8121137c64e253d026316bc0f6ead2954b96f326dbe9b436ee8d49415de"
Jul 17 10:15:31.201020 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:31.200987    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72" event={"ID":"e47ec946-8125-4f2c-8170-05af5f43dd87","Type":"ContainerStarted","Data":"76e9736a5f200c95841c1a725267db77c58ba077e07ad281d300711b996d9e17"}
Jul 17 10:15:31.225545 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:31.225495    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72" podStartSLOduration=18.225478102 podStartE2EDuration="18.225478102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:31.224296875 +0000 UTC m=+3415.289117599" watchObservedRunningTime="2026-07-17 10:15:31.225478102 +0000 UTC m=+3415.290298815"
Jul 17 10:15:32.208138 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:32.208099    2571 generic.go:356] "Generic (PLEG): container finished" podID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerID="76e9736a5f200c95841c1a725267db77c58ba077e07ad281d300711b996d9e17" exitCode=6
Jul 17 10:15:32.208652 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:32.208180    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72" event={"ID":"e47ec946-8125-4f2c-8170-05af5f43dd87","Type":"ContainerDied","Data":"76e9736a5f200c95841c1a725267db77c58ba077e07ad281d300711b996d9e17"}
Jul 17 10:15:32.208652 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:32.208260    2571 scope.go:117] "RemoveContainer" containerID="dd5cd8121137c64e253d026316bc0f6ead2954b96f326dbe9b436ee8d49415de"
Jul 17 10:15:32.208652 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:32.208571    2571 scope.go:117] "RemoveContainer" containerID="76e9736a5f200c95841c1a725267db77c58ba077e07ad281d300711b996d9e17"
Jul 17 10:15:32.208869 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:32.208847    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72_odh-ai-gateway-infra(e47ec946-8125-4f2c-8170-05af5f43dd87)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87"
Jul 17 10:15:32.244542 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:32.244512    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"]
Jul 17 10:15:33.356563 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:33.356538    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"
Jul 17 10:15:33.436680 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:33.436644    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e47ec946-8125-4f2c-8170-05af5f43dd87-maas-api-service-ca\") pod \"e47ec946-8125-4f2c-8170-05af5f43dd87\" (UID: \"e47ec946-8125-4f2c-8170-05af5f43dd87\") "
Jul 17 10:15:33.437051 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:33.437023    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e47ec946-8125-4f2c-8170-05af5f43dd87-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e47ec946-8125-4f2c-8170-05af5f43dd87" (UID: "e47ec946-8125-4f2c-8170-05af5f43dd87"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:33.537925 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:33.537839    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e47ec946-8125-4f2c-8170-05af5f43dd87-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:15:34.223742 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:34.223704    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"
Jul 17 10:15:34.223907 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:34.223756    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72" event={"ID":"e47ec946-8125-4f2c-8170-05af5f43dd87","Type":"ContainerDied","Data":"8612fef9b08942d2e3c71a288747fd14bb504b80db3dc09427851be5fbe245a9"}
Jul 17 10:15:34.223907 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:34.223820    2571 scope.go:117] "RemoveContainer" containerID="76e9736a5f200c95841c1a725267db77c58ba077e07ad281d300711b996d9e17"
Jul 17 10:15:34.251417 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:34.251383    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"]
Jul 17 10:15:34.256589 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:34.256566    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-kxh72"]
Jul 17 10:15:35.028093 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.028058    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"]
Jul 17 10:15:35.028549 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.028534    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerName="revoke-keys"
Jul 17 10:15:35.028601 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.028551    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerName="revoke-keys"
Jul 17 10:15:35.028601 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.028561    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerName="revoke-keys"
Jul 17 10:15:35.028601 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.028567    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerName="revoke-keys"
Jul 17 10:15:35.028601 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.028577    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerName="revoke-keys"
Jul 17 10:15:35.028601 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.028583    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerName="revoke-keys"
Jul 17 10:15:35.028761 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.028656    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerName="revoke-keys"
Jul 17 10:15:35.028761 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.028665    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerName="revoke-keys"
Jul 17 10:15:35.028761 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.028673    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="26518bae-0854-40a0-9cf9-8c6377e623a2" containerName="revoke-keys"
Jul 17 10:15:35.033313 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.033290    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"
Jul 17 10:15:35.042425 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.042393    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"]
Jul 17 10:15:35.153553 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.153501    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2bc83ec-22b4-458f-ba31-1e4f89c6305e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts\" (UID: \"c2bc83ec-22b4-458f-ba31-1e4f89c6305e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"
Jul 17 10:15:35.255027 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.254994    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2bc83ec-22b4-458f-ba31-1e4f89c6305e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts\" (UID: \"c2bc83ec-22b4-458f-ba31-1e4f89c6305e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"
Jul 17 10:15:35.255701 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.255680    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2bc83ec-22b4-458f-ba31-1e4f89c6305e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts\" (UID: \"c2bc83ec-22b4-458f-ba31-1e4f89c6305e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"
Jul 17 10:15:35.345203 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.345119    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"
Jul 17 10:15:35.483191 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.483160    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"]
Jul 17 10:15:35.483754 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:15:35.483714    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2bc83ec_22b4_458f_ba31_1e4f89c6305e.slice/crio-c048a9ebe9f10092b47a26fdb23fab653dab1cbefc9375b8546ae1b56bd80704 WatchSource:0}: Error finding container c048a9ebe9f10092b47a26fdb23fab653dab1cbefc9375b8546ae1b56bd80704: Status 404 returned error can't find the container with id c048a9ebe9f10092b47a26fdb23fab653dab1cbefc9375b8546ae1b56bd80704
Jul 17 10:15:35.758370 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:35.758338    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87" path="/var/lib/kubelet/pods/e47ec946-8125-4f2c-8170-05af5f43dd87/volumes"
Jul 17 10:15:36.239535 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:36.239503    2571 generic.go:356] "Generic (PLEG): container finished" podID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerID="ade4afaa90696601344cc5a1aa71c74a50b5296ef04b0cd69f7bd9c4291a5fd1" exitCode=6
Jul 17 10:15:36.239931 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:36.239600    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts" event={"ID":"c2bc83ec-22b4-458f-ba31-1e4f89c6305e","Type":"ContainerDied","Data":"ade4afaa90696601344cc5a1aa71c74a50b5296ef04b0cd69f7bd9c4291a5fd1"}
Jul 17 10:15:36.239931 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:36.239663    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts" event={"ID":"c2bc83ec-22b4-458f-ba31-1e4f89c6305e","Type":"ContainerStarted","Data":"c048a9ebe9f10092b47a26fdb23fab653dab1cbefc9375b8546ae1b56bd80704"}
Jul 17 10:15:36.239931 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:36.239786    2571 scope.go:117] "RemoveContainer" containerID="ade4afaa90696601344cc5a1aa71c74a50b5296ef04b0cd69f7bd9c4291a5fd1"
Jul 17 10:15:37.247067 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:37.247036    2571 generic.go:356] "Generic (PLEG): container finished" podID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerID="f6bdda85da03547ec7503df11ca15c330e33dbc17cd9f618eac14c4191d4087e" exitCode=6
Jul 17 10:15:37.247560 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:37.247125    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts" event={"ID":"c2bc83ec-22b4-458f-ba31-1e4f89c6305e","Type":"ContainerDied","Data":"f6bdda85da03547ec7503df11ca15c330e33dbc17cd9f618eac14c4191d4087e"}
Jul 17 10:15:37.247560 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:37.247183    2571 scope.go:117] "RemoveContainer" containerID="ade4afaa90696601344cc5a1aa71c74a50b5296ef04b0cd69f7bd9c4291a5fd1"
Jul 17 10:15:37.247560 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:37.247407    2571 scope.go:117] "RemoveContainer" containerID="f6bdda85da03547ec7503df11ca15c330e33dbc17cd9f618eac14c4191d4087e"
Jul 17 10:15:37.247748 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:37.247663    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts_odh-ai-gateway-infra(c2bc83ec-22b4-458f-ba31-1e4f89c6305e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e"
Jul 17 10:15:38.153676 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:38.153643    2571 scope.go:117] "RemoveContainer" containerID="320450a7d1e6979bef5bd9ffd9e005a14c0b1831595973151293fc9a5f412dd3"
Jul 17 10:15:38.254330 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:38.254305    2571 scope.go:117] "RemoveContainer" containerID="f6bdda85da03547ec7503df11ca15c330e33dbc17cd9f618eac14c4191d4087e"
Jul 17 10:15:38.254716 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:38.254513    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts_odh-ai-gateway-infra(c2bc83ec-22b4-458f-ba31-1e4f89c6305e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e"
Jul 17 10:15:39.672719 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:39.672683    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7220-139733724386224"]
Jul 17 10:15:39.673241 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:39.673228    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e47ec946-8125-4f2c-8170-05af5f43dd87" containerName="revoke-keys"
Jul 17 10:15:39.677413 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:39.677396    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724386224"
Jul 17 10:15:39.680093 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:39.680069    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-8l869\""
Jul 17 10:15:39.683144 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:39.683115    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7220-139733724386224"]
Jul 17 10:15:39.795880 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:39.795843    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89xqq\" (UniqueName: \"kubernetes.io/projected/f12acc89-3524-4eca-b402-ea3bf8644a32-kube-api-access-89xqq\") pod \"test-curl-7220-139733724386224\" (UID: \"f12acc89-3524-4eca-b402-ea3bf8644a32\") " pod="opendatahub/test-curl-7220-139733724386224"
Jul 17 10:15:39.897185 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:39.897145    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89xqq\" (UniqueName: \"kubernetes.io/projected/f12acc89-3524-4eca-b402-ea3bf8644a32-kube-api-access-89xqq\") pod \"test-curl-7220-139733724386224\" (UID: \"f12acc89-3524-4eca-b402-ea3bf8644a32\") " pod="opendatahub/test-curl-7220-139733724386224"
Jul 17 10:15:39.906116 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:39.906087    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89xqq\" (UniqueName: \"kubernetes.io/projected/f12acc89-3524-4eca-b402-ea3bf8644a32-kube-api-access-89xqq\") pod \"test-curl-7220-139733724386224\" (UID: \"f12acc89-3524-4eca-b402-ea3bf8644a32\") " pod="opendatahub/test-curl-7220-139733724386224"
Jul 17 10:15:39.988762 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:39.988722    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724386224"
Jul 17 10:15:40.129039 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:40.129011    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7220-139733724386224"]
Jul 17 10:15:40.130749 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:15:40.130720    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf12acc89_3524_4eca_b402_ea3bf8644a32.slice/crio-cc25e8c8d30b46356f505aa021d0b9a3e0d223c4621cb04543715fb0b44b28ef WatchSource:0}: Error finding container cc25e8c8d30b46356f505aa021d0b9a3e0d223c4621cb04543715fb0b44b28ef: Status 404 returned error can't find the container with id cc25e8c8d30b46356f505aa021d0b9a3e0d223c4621cb04543715fb0b44b28ef
Jul 17 10:15:40.267695 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:40.267613    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7220-139733724386224" event={"ID":"f12acc89-3524-4eca-b402-ea3bf8644a32","Type":"ContainerStarted","Data":"cc25e8c8d30b46356f505aa021d0b9a3e0d223c4621cb04543715fb0b44b28ef"}
Jul 17 10:15:41.275048 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:41.275011    2571 generic.go:356] "Generic (PLEG): container finished" podID="f12acc89-3524-4eca-b402-ea3bf8644a32" containerID="cad104936457fb437dbdbcc36e456970ca1ee9c9ec85faf80923f08a7c886af9" exitCode=0
Jul 17 10:15:41.275443 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:41.275063    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7220-139733724386224" event={"ID":"f12acc89-3524-4eca-b402-ea3bf8644a32","Type":"ContainerDied","Data":"cad104936457fb437dbdbcc36e456970ca1ee9c9ec85faf80923f08a7c886af9"}
Jul 17 10:15:42.422339 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:42.422314    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724386224"
Jul 17 10:15:42.465988 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:42.465955    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7220-139733724386224_f12acc89-3524-4eca-b402-ea3bf8644a32/test-curl-7220-139733724386224/0.log"
Jul 17 10:15:42.490486 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:42.490428    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7220-139733724386224"]
Jul 17 10:15:42.494475 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:42.494441    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7220-139733724386224"]
Jul 17 10:15:42.522533 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:42.522502    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89xqq\" (UniqueName: \"kubernetes.io/projected/f12acc89-3524-4eca-b402-ea3bf8644a32-kube-api-access-89xqq\") pod \"f12acc89-3524-4eca-b402-ea3bf8644a32\" (UID: \"f12acc89-3524-4eca-b402-ea3bf8644a32\") "
Jul 17 10:15:42.524767 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:42.524729    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12acc89-3524-4eca-b402-ea3bf8644a32-kube-api-access-89xqq" (OuterVolumeSpecName: "kube-api-access-89xqq") pod "f12acc89-3524-4eca-b402-ea3bf8644a32" (UID: "f12acc89-3524-4eca-b402-ea3bf8644a32"). InnerVolumeSpecName "kube-api-access-89xqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:15:42.623852 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:42.623780    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89xqq\" (UniqueName: \"kubernetes.io/projected/f12acc89-3524-4eca-b402-ea3bf8644a32-kube-api-access-89xqq\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:15:42.752551 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:42.752518    2571 scope.go:117] "RemoveContainer" containerID="dcc9f76b550a5f7f6f023a787dbdd4442929e81dc9da78f0514701d3d7a2547e"
Jul 17 10:15:43.289254 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:43.289157    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh" event={"ID":"f5aca8b8-b6e2-4072-93db-6498e7f85203","Type":"ContainerStarted","Data":"ed126c18dda7137787facb1cdd07dfb241db73d2eb09feafb2da9d2c6d2c932f"}
Jul 17 10:15:43.290603 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:43.290577    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7220-139733724386224"
Jul 17 10:15:43.290747 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:43.290586    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc25e8c8d30b46356f505aa021d0b9a3e0d223c4621cb04543715fb0b44b28ef"
Jul 17 10:15:43.316162 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:43.316102    2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7220-139733724386224\" is forbidden: User \"system:node:ip-10-0-138-208.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-208.ec2.internal' and this object" podUID="f12acc89-3524-4eca-b402-ea3bf8644a32" pod="opendatahub/test-curl-7220-139733724386224"
Jul 17 10:15:43.316320 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:43.316169    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh" podStartSLOduration=17.316155696 podStartE2EDuration="17.316155696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:43.313308536 +0000 UTC m=+3427.378129255" watchObservedRunningTime="2026-07-17 10:15:43.316155696 +0000 UTC m=+3427.380976429"
Jul 17 10:15:43.759238 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:43.759203    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f12acc89-3524-4eca-b402-ea3bf8644a32" path="/var/lib/kubelet/pods/f12acc89-3524-4eca-b402-ea3bf8644a32/volumes"
Jul 17 10:15:44.298094 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:44.298063    2571 generic.go:356] "Generic (PLEG): container finished" podID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerID="ed126c18dda7137787facb1cdd07dfb241db73d2eb09feafb2da9d2c6d2c932f" exitCode=6
Jul 17 10:15:44.298296 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:44.298141    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh" event={"ID":"f5aca8b8-b6e2-4072-93db-6498e7f85203","Type":"ContainerDied","Data":"ed126c18dda7137787facb1cdd07dfb241db73d2eb09feafb2da9d2c6d2c932f"}
Jul 17 10:15:44.298296 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:44.298216    2571 scope.go:117] "RemoveContainer" containerID="dcc9f76b550a5f7f6f023a787dbdd4442929e81dc9da78f0514701d3d7a2547e"
Jul 17 10:15:44.298565 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:44.298547    2571 scope.go:117] "RemoveContainer" containerID="ed126c18dda7137787facb1cdd07dfb241db73d2eb09feafb2da9d2c6d2c932f"
Jul 17 10:15:44.298791 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:44.298766    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh_odh-ai-gateway-infra(f5aca8b8-b6e2-4072-93db-6498e7f85203)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203"
Jul 17 10:15:44.325078 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:44.325050    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"]
Jul 17 10:15:45.442727 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:45.442699    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"
Jul 17 10:15:45.553244 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:45.553202    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5aca8b8-b6e2-4072-93db-6498e7f85203-maas-api-service-ca\") pod \"f5aca8b8-b6e2-4072-93db-6498e7f85203\" (UID: \"f5aca8b8-b6e2-4072-93db-6498e7f85203\") "
Jul 17 10:15:45.553596 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:45.553574    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5aca8b8-b6e2-4072-93db-6498e7f85203-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f5aca8b8-b6e2-4072-93db-6498e7f85203" (UID: "f5aca8b8-b6e2-4072-93db-6498e7f85203"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:45.653962 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:45.653876    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5aca8b8-b6e2-4072-93db-6498e7f85203-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:15:46.313099 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:46.313061    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh" event={"ID":"f5aca8b8-b6e2-4072-93db-6498e7f85203","Type":"ContainerDied","Data":"6fab43dffe8b76807f1d3f4c38b736fa77060b5d944963ba76e69c5b69aaf64a"}
Jul 17 10:15:46.313099 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:46.313098    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"
Jul 17 10:15:46.313345 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:46.313136    2571 scope.go:117] "RemoveContainer" containerID="ed126c18dda7137787facb1cdd07dfb241db73d2eb09feafb2da9d2c6d2c932f"
Jul 17 10:15:46.351219 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:46.351190    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"]
Jul 17 10:15:46.358280 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:46.358251    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-wd6sh"]
Jul 17 10:15:47.758687 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:47.758654    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203" path="/var/lib/kubelet/pods/f5aca8b8-b6e2-4072-93db-6498e7f85203/volumes"
Jul 17 10:15:49.758721 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:49.758690    2571 scope.go:117] "RemoveContainer" containerID="f6bdda85da03547ec7503df11ca15c330e33dbc17cd9f618eac14c4191d4087e"
Jul 17 10:15:50.340739 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:50.340700    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts" event={"ID":"c2bc83ec-22b4-458f-ba31-1e4f89c6305e","Type":"ContainerStarted","Data":"fb498f7ba6f3c86f545c880d30755cb86c77a76bbe5968e9b43f734124537f99"}
Jul 17 10:15:50.365718 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:50.365667    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts" podStartSLOduration=15.365650902 podStartE2EDuration="15.365650902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:50.362820848 +0000 UTC m=+3434.427641571" watchObservedRunningTime="2026-07-17 10:15:50.365650902 +0000 UTC m=+3434.430471650"
Jul 17 10:15:51.348601 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:51.348567    2571 generic.go:356] "Generic (PLEG): container finished" podID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerID="fb498f7ba6f3c86f545c880d30755cb86c77a76bbe5968e9b43f734124537f99" exitCode=6
Jul 17 10:15:51.349008 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:51.348639    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts" event={"ID":"c2bc83ec-22b4-458f-ba31-1e4f89c6305e","Type":"ContainerDied","Data":"fb498f7ba6f3c86f545c880d30755cb86c77a76bbe5968e9b43f734124537f99"}
Jul 17 10:15:51.349008 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:51.348704    2571 scope.go:117] "RemoveContainer" containerID="f6bdda85da03547ec7503df11ca15c330e33dbc17cd9f618eac14c4191d4087e"
Jul 17 10:15:51.349008 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:51.348971    2571 scope.go:117] "RemoveContainer" containerID="fb498f7ba6f3c86f545c880d30755cb86c77a76bbe5968e9b43f734124537f99"
Jul 17 10:15:51.349235 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:51.349216    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts_odh-ai-gateway-infra(c2bc83ec-22b4-458f-ba31-1e4f89c6305e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e"
Jul 17 10:15:51.381004 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:51.380973    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"]
Jul 17 10:15:52.499426 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:52.499399    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"
Jul 17 10:15:52.615389 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:52.615298    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2bc83ec-22b4-458f-ba31-1e4f89c6305e-maas-api-service-ca\") pod \"c2bc83ec-22b4-458f-ba31-1e4f89c6305e\" (UID: \"c2bc83ec-22b4-458f-ba31-1e4f89c6305e\") "
Jul 17 10:15:52.615770 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:52.615749    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2bc83ec-22b4-458f-ba31-1e4f89c6305e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c2bc83ec-22b4-458f-ba31-1e4f89c6305e" (UID: "c2bc83ec-22b4-458f-ba31-1e4f89c6305e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:52.717028 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:52.716992    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2bc83ec-22b4-458f-ba31-1e4f89c6305e-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:15:53.364307 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:53.364274    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"
Jul 17 10:15:53.364487 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:53.364328    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts" event={"ID":"c2bc83ec-22b4-458f-ba31-1e4f89c6305e","Type":"ContainerDied","Data":"c048a9ebe9f10092b47a26fdb23fab653dab1cbefc9375b8546ae1b56bd80704"}
Jul 17 10:15:53.364487 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:53.364383    2571 scope.go:117] "RemoveContainer" containerID="fb498f7ba6f3c86f545c880d30755cb86c77a76bbe5968e9b43f734124537f99"
Jul 17 10:15:53.390574 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:53.390541    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"]
Jul 17 10:15:53.401643 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:53.401609    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-m2fts"]
Jul 17 10:15:53.758867 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:53.758835    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" path="/var/lib/kubelet/pods/c2bc83ec-22b4-458f-ba31-1e4f89c6305e/volumes"
Jul 17 10:15:54.525453 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525417    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786"]
Jul 17 10:15:54.525897 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525884    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerName="revoke-keys"
Jul 17 10:15:54.525960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525899    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerName="revoke-keys"
Jul 17 10:15:54.525960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525906    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12acc89-3524-4eca-b402-ea3bf8644a32" containerName="test-curl-7220-139733724386224"
Jul 17 10:15:54.525960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525913    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12acc89-3524-4eca-b402-ea3bf8644a32" containerName="test-curl-7220-139733724386224"
Jul 17 10:15:54.525960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525921    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerName="revoke-keys"
Jul 17 10:15:54.525960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525926    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerName="revoke-keys"
Jul 17 10:15:54.525960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525937    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerName="revoke-keys"
Jul 17 10:15:54.525960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525943    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerName="revoke-keys"
Jul 17 10:15:54.525960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525950    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerName="revoke-keys"
Jul 17 10:15:54.525960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525955    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerName="revoke-keys"
Jul 17 10:15:54.525960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525963    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerName="revoke-keys"
Jul 17 10:15:54.526287 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.525970    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerName="revoke-keys"
Jul 17 10:15:54.526287 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.526037    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerName="revoke-keys"
Jul 17 10:15:54.526287 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.526044    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f12acc89-3524-4eca-b402-ea3bf8644a32" containerName="test-curl-7220-139733724386224"
Jul 17 10:15:54.526287 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.526051    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerName="revoke-keys"
Jul 17 10:15:54.526287 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.526056    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2bc83ec-22b4-458f-ba31-1e4f89c6305e" containerName="revoke-keys"
Jul 17 10:15:54.526287 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.526063    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerName="revoke-keys"
Jul 17 10:15:54.526287 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.526069    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerName="revoke-keys"
Jul 17 10:15:54.526287 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.526074    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerName="revoke-keys"
Jul 17 10:15:54.530203 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.530186    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786"
Jul 17 10:15:54.532958 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.532937    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:15:54.540717 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.540688    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786"]
Jul 17 10:15:54.632770 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.632732    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51f2b841-3aa2-4bd3-b0b1-01108304b63a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786\" (UID: \"51f2b841-3aa2-4bd3-b0b1-01108304b63a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786"
Jul 17 10:15:54.733986 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.733952    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51f2b841-3aa2-4bd3-b0b1-01108304b63a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786\" (UID: \"51f2b841-3aa2-4bd3-b0b1-01108304b63a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786"
Jul 17 10:15:54.734676 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.734654    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51f2b841-3aa2-4bd3-b0b1-01108304b63a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786\" (UID: \"51f2b841-3aa2-4bd3-b0b1-01108304b63a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786"
Jul 17 10:15:54.845023 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:54.844941    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786"
Jul 17 10:15:55.214457 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:15:55.214427    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51f2b841_3aa2_4bd3_b0b1_01108304b63a.slice/crio-d61050592c419976a47b1ecf5a88ef3eec69608496aae29583bb480cc9a91c60 WatchSource:0}: Error finding container d61050592c419976a47b1ecf5a88ef3eec69608496aae29583bb480cc9a91c60: Status 404 returned error can't find the container with id d61050592c419976a47b1ecf5a88ef3eec69608496aae29583bb480cc9a91c60
Jul 17 10:15:55.215058 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:55.215034    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786"]
Jul 17 10:15:55.380544 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:55.380498    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786" event={"ID":"51f2b841-3aa2-4bd3-b0b1-01108304b63a","Type":"ContainerStarted","Data":"d61050592c419976a47b1ecf5a88ef3eec69608496aae29583bb480cc9a91c60"}
Jul 17 10:15:56.388712 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.388679    2571 generic.go:356] "Generic (PLEG): container finished" podID="51f2b841-3aa2-4bd3-b0b1-01108304b63a" containerID="15e217adbb8d733f0e9b57d96f36c0d9a84a65919288202ef775fe0af33d0f87" exitCode=0
Jul 17 10:15:56.389091 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.388765    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786" event={"ID":"51f2b841-3aa2-4bd3-b0b1-01108304b63a","Type":"ContainerDied","Data":"15e217adbb8d733f0e9b57d96f36c0d9a84a65919288202ef775fe0af33d0f87"}
Jul 17 10:15:56.475604 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.475572    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"]
Jul 17 10:15:56.476045 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.476032    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerName="revoke-keys"
Jul 17 10:15:56.476094 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.476047    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5aca8b8-b6e2-4072-93db-6498e7f85203" containerName="revoke-keys"
Jul 17 10:15:56.479405 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.479387    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"
Jul 17 10:15:56.488673 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.488646    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"]
Jul 17 10:15:56.649293 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.649202    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85103364-7a42-424a-a9cb-7ff8b32f3577-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk\" (UID: \"85103364-7a42-424a-a9cb-7ff8b32f3577\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"
Jul 17 10:15:56.749950 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.749904    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85103364-7a42-424a-a9cb-7ff8b32f3577-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk\" (UID: \"85103364-7a42-424a-a9cb-7ff8b32f3577\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"
Jul 17 10:15:56.750696 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.750670    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85103364-7a42-424a-a9cb-7ff8b32f3577-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk\" (UID: \"85103364-7a42-424a-a9cb-7ff8b32f3577\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"
Jul 17 10:15:56.790482 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.790438    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"
Jul 17 10:15:56.938593 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:56.938568    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"]
Jul 17 10:15:56.940514 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:15:56.940486    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85103364_7a42_424a_a9cb_7ff8b32f3577.slice/crio-8f89d1037ca2921839a005a466d49e268a6d4d5447ac397fb30f62d670ed9851 WatchSource:0}: Error finding container 8f89d1037ca2921839a005a466d49e268a6d4d5447ac397fb30f62d670ed9851: Status 404 returned error can't find the container with id 8f89d1037ca2921839a005a466d49e268a6d4d5447ac397fb30f62d670ed9851
Jul 17 10:15:57.397539 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:57.397500    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk" event={"ID":"85103364-7a42-424a-a9cb-7ff8b32f3577","Type":"ContainerStarted","Data":"73451113a54f40c14b971c3a614ad8cff58dad155a9088c66f48989f58e7dab0"}
Jul 17 10:15:57.397937 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:57.397558    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk" event={"ID":"85103364-7a42-424a-a9cb-7ff8b32f3577","Type":"ContainerStarted","Data":"8f89d1037ca2921839a005a466d49e268a6d4d5447ac397fb30f62d670ed9851"}
Jul 17 10:15:57.428391 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:57.428325    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk" podStartSLOduration=1.428305097 podStartE2EDuration="1.428305097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:15:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:15:57.42576217 +0000 UTC m=+3441.490582892" watchObservedRunningTime="2026-07-17 10:15:57.428305097 +0000 UTC m=+3441.493125820"
Jul 17 10:15:57.538853 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:57.538828    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786"
Jul 17 10:15:57.656923 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:57.656898    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51f2b841-3aa2-4bd3-b0b1-01108304b63a-maas-api-service-ca\") pod \"51f2b841-3aa2-4bd3-b0b1-01108304b63a\" (UID: \"51f2b841-3aa2-4bd3-b0b1-01108304b63a\") "
Jul 17 10:15:57.657217 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:57.657190    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51f2b841-3aa2-4bd3-b0b1-01108304b63a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "51f2b841-3aa2-4bd3-b0b1-01108304b63a" (UID: "51f2b841-3aa2-4bd3-b0b1-01108304b63a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:15:57.758300 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:57.758269    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51f2b841-3aa2-4bd3-b0b1-01108304b63a-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:15:58.404915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:58.404881    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786"
Jul 17 10:15:58.404915 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:58.404908    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786" event={"ID":"51f2b841-3aa2-4bd3-b0b1-01108304b63a","Type":"ContainerDied","Data":"d61050592c419976a47b1ecf5a88ef3eec69608496aae29583bb480cc9a91c60"}
Jul 17 10:15:58.405368 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:58.404961    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d61050592c419976a47b1ecf5a88ef3eec69608496aae29583bb480cc9a91c60"
Jul 17 10:15:58.406593 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:58.406552    2571 generic.go:356] "Generic (PLEG): container finished" podID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerID="73451113a54f40c14b971c3a614ad8cff58dad155a9088c66f48989f58e7dab0" exitCode=6
Jul 17 10:15:58.406742 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:58.406591    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk" event={"ID":"85103364-7a42-424a-a9cb-7ff8b32f3577","Type":"ContainerDied","Data":"73451113a54f40c14b971c3a614ad8cff58dad155a9088c66f48989f58e7dab0"}
Jul 17 10:15:58.406870 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:58.406852    2571 scope.go:117] "RemoveContainer" containerID="73451113a54f40c14b971c3a614ad8cff58dad155a9088c66f48989f58e7dab0"
Jul 17 10:15:59.415504 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:59.415410    2571 generic.go:356] "Generic (PLEG): container finished" podID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerID="ea09dccd484ea1ad26216f4e3ee78da2ba66ab11d920be7d5f635986cb8764ad" exitCode=6
Jul 17 10:15:59.415902 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:59.415505    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk" event={"ID":"85103364-7a42-424a-a9cb-7ff8b32f3577","Type":"ContainerDied","Data":"ea09dccd484ea1ad26216f4e3ee78da2ba66ab11d920be7d5f635986cb8764ad"}
Jul 17 10:15:59.415902 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:59.415566    2571 scope.go:117] "RemoveContainer" containerID="73451113a54f40c14b971c3a614ad8cff58dad155a9088c66f48989f58e7dab0"
Jul 17 10:15:59.416014 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:15:59.415910    2571 scope.go:117] "RemoveContainer" containerID="ea09dccd484ea1ad26216f4e3ee78da2ba66ab11d920be7d5f635986cb8764ad"
Jul 17 10:15:59.416158 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:15:59.416141    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk_odh-ai-gateway-infra(85103364-7a42-424a-a9cb-7ff8b32f3577)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577"
Jul 17 10:16:00.422824 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:00.422796    2571 scope.go:117] "RemoveContainer" containerID="ea09dccd484ea1ad26216f4e3ee78da2ba66ab11d920be7d5f635986cb8764ad"
Jul 17 10:16:00.423190 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:16:00.422981    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk_odh-ai-gateway-infra(85103364-7a42-424a-a9cb-7ff8b32f3577)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577"
Jul 17 10:16:05.169083 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:05.169041    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl"]
Jul 17 10:16:05.178694 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:05.178652    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-8ae1eb-29738055-m4fjl"]
Jul 17 10:16:05.759137 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:05.759103    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9479e35-51f9-42f8-a00f-1d857d5de9bb" path="/var/lib/kubelet/pods/c9479e35-51f9-42f8-a00f-1d857d5de9bb/volumes"
Jul 17 10:16:11.753062 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:11.753021    2571 scope.go:117] "RemoveContainer" containerID="ea09dccd484ea1ad26216f4e3ee78da2ba66ab11d920be7d5f635986cb8764ad"
Jul 17 10:16:12.496667 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:12.496637    2571 generic.go:356] "Generic (PLEG): container finished" podID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerID="34e6ef62aae3b327a54a32fa4a12f07e81867249097d66cfe42835a1697b2775" exitCode=6
Jul 17 10:16:12.496833 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:12.496712    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk" event={"ID":"85103364-7a42-424a-a9cb-7ff8b32f3577","Type":"ContainerDied","Data":"34e6ef62aae3b327a54a32fa4a12f07e81867249097d66cfe42835a1697b2775"}
Jul 17 10:16:12.496833 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:12.496781    2571 scope.go:117] "RemoveContainer" containerID="ea09dccd484ea1ad26216f4e3ee78da2ba66ab11d920be7d5f635986cb8764ad"
Jul 17 10:16:12.497089 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:12.497054    2571 scope.go:117] "RemoveContainer" containerID="34e6ef62aae3b327a54a32fa4a12f07e81867249097d66cfe42835a1697b2775"
Jul 17 10:16:12.497321 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:16:12.497302    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk_odh-ai-gateway-infra(85103364-7a42-424a-a9cb-7ff8b32f3577)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577"
Jul 17 10:16:13.536351 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.536316    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"]
Jul 17 10:16:13.686015 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.685986    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"
Jul 17 10:16:13.883162 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.883068    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85103364-7a42-424a-a9cb-7ff8b32f3577-maas-api-service-ca\") pod \"85103364-7a42-424a-a9cb-7ff8b32f3577\" (UID: \"85103364-7a42-424a-a9cb-7ff8b32f3577\") "
Jul 17 10:16:13.883430 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.883407    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85103364-7a42-424a-a9cb-7ff8b32f3577-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85103364-7a42-424a-a9cb-7ff8b32f3577" (UID: "85103364-7a42-424a-a9cb-7ff8b32f3577"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:13.960389 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.960349    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"]
Jul 17 10:16:13.961018 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.961001    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51f2b841-3aa2-4bd3-b0b1-01108304b63a" containerName="revoke-keys"
Jul 17 10:16:13.961064 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.961022    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="51f2b841-3aa2-4bd3-b0b1-01108304b63a" containerName="revoke-keys"
Jul 17 10:16:13.961064 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.961035    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerName="revoke-keys"
Jul 17 10:16:13.961064 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.961043    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerName="revoke-keys"
Jul 17 10:16:13.961156 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.961080    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerName="revoke-keys"
Jul 17 10:16:13.961156 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.961089    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerName="revoke-keys"
Jul 17 10:16:13.961222 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.961180    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerName="revoke-keys"
Jul 17 10:16:13.961222 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.961195    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="51f2b841-3aa2-4bd3-b0b1-01108304b63a" containerName="revoke-keys"
Jul 17 10:16:13.961222 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.961208    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerName="revoke-keys"
Jul 17 10:16:13.965433 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.965413    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"
Jul 17 10:16:13.973073 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.973047    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"]
Jul 17 10:16:13.983867 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.983817    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94e0cf34-9d7e-47d1-8e6c-ae890251b456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp\" (UID: \"94e0cf34-9d7e-47d1-8e6c-ae890251b456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"
Jul 17 10:16:13.984039 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:13.983936    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85103364-7a42-424a-a9cb-7ff8b32f3577-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:16:14.084393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:14.084349    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94e0cf34-9d7e-47d1-8e6c-ae890251b456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp\" (UID: \"94e0cf34-9d7e-47d1-8e6c-ae890251b456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"
Jul 17 10:16:14.085018 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:14.085000    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94e0cf34-9d7e-47d1-8e6c-ae890251b456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp\" (UID: \"94e0cf34-9d7e-47d1-8e6c-ae890251b456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"
Jul 17 10:16:14.278841 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:14.278786    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"
Jul 17 10:16:14.511287 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:14.511248    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk" event={"ID":"85103364-7a42-424a-a9cb-7ff8b32f3577","Type":"ContainerDied","Data":"8f89d1037ca2921839a005a466d49e268a6d4d5447ac397fb30f62d670ed9851"}
Jul 17 10:16:14.511487 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:14.511323    2571 scope.go:117] "RemoveContainer" containerID="34e6ef62aae3b327a54a32fa4a12f07e81867249097d66cfe42835a1697b2775"
Jul 17 10:16:14.511487 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:14.511259    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"
Jul 17 10:16:14.540063 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:14.539992    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"]
Jul 17 10:16:14.542636 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:14.542606    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f15c7b6a-vwggk"]
Jul 17 10:16:14.627357 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:14.627332    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"]
Jul 17 10:16:14.629572 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:16:14.629540    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94e0cf34_9d7e_47d1_8e6c_ae890251b456.slice/crio-889211cea9a231225e5f7250d71ab95fa44c002942371390a70daf08b79231a2 WatchSource:0}: Error finding container 889211cea9a231225e5f7250d71ab95fa44c002942371390a70daf08b79231a2: Status 404 returned error can't find the container with id 889211cea9a231225e5f7250d71ab95fa44c002942371390a70daf08b79231a2
Jul 17 10:16:15.526992 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:15.526959    2571 generic.go:356] "Generic (PLEG): container finished" podID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerID="e9129e7c9c8a5e764f43c5c7f07051f4a6729b894d954a8d553694317ecbc71c" exitCode=6
Jul 17 10:16:15.527160 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:15.527032    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp" event={"ID":"94e0cf34-9d7e-47d1-8e6c-ae890251b456","Type":"ContainerDied","Data":"e9129e7c9c8a5e764f43c5c7f07051f4a6729b894d954a8d553694317ecbc71c"}
Jul 17 10:16:15.527160 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:15.527090    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp" event={"ID":"94e0cf34-9d7e-47d1-8e6c-ae890251b456","Type":"ContainerStarted","Data":"889211cea9a231225e5f7250d71ab95fa44c002942371390a70daf08b79231a2"}
Jul 17 10:16:15.527284 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:15.527272    2571 scope.go:117] "RemoveContainer" containerID="e9129e7c9c8a5e764f43c5c7f07051f4a6729b894d954a8d553694317ecbc71c"
Jul 17 10:16:15.758777 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:15.758749    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577" path="/var/lib/kubelet/pods/85103364-7a42-424a-a9cb-7ff8b32f3577/volumes"
Jul 17 10:16:16.535097 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:16.535056    2571 generic.go:356] "Generic (PLEG): container finished" podID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerID="68f9c4454e21fab2223c9f4e727a8f83849a26308007a9cdfbe66617e4a9fd06" exitCode=6
Jul 17 10:16:16.535277 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:16.535100    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp" event={"ID":"94e0cf34-9d7e-47d1-8e6c-ae890251b456","Type":"ContainerDied","Data":"68f9c4454e21fab2223c9f4e727a8f83849a26308007a9cdfbe66617e4a9fd06"}
Jul 17 10:16:16.535277 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:16.535163    2571 scope.go:117] "RemoveContainer" containerID="e9129e7c9c8a5e764f43c5c7f07051f4a6729b894d954a8d553694317ecbc71c"
Jul 17 10:16:16.535424 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:16.535371    2571 scope.go:117] "RemoveContainer" containerID="68f9c4454e21fab2223c9f4e727a8f83849a26308007a9cdfbe66617e4a9fd06"
Jul 17 10:16:16.535649 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:16:16.535625    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp_odh-ai-gateway-infra(94e0cf34-9d7e-47d1-8e6c-ae890251b456)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456"
Jul 17 10:16:17.543225 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:17.543197    2571 scope.go:117] "RemoveContainer" containerID="68f9c4454e21fab2223c9f4e727a8f83849a26308007a9cdfbe66617e4a9fd06"
Jul 17 10:16:17.543635 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:16:17.543407    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp_odh-ai-gateway-infra(94e0cf34-9d7e-47d1-8e6c-ae890251b456)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456"
Jul 17 10:16:20.723878 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:20.723842    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d"]
Jul 17 10:16:20.724548 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:20.724528    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerName="revoke-keys"
Jul 17 10:16:20.724548 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:20.724550    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerName="revoke-keys"
Jul 17 10:16:20.724719 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:20.724647    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="85103364-7a42-424a-a9cb-7ff8b32f3577" containerName="revoke-keys"
Jul 17 10:16:20.727388 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:20.727365    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d"
Jul 17 10:16:20.736814 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:20.736789    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d"]
Jul 17 10:16:20.738995 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:20.738972    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7918923-bcfc-4138-8f05-ba118fd32e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d\" (UID: \"b7918923-bcfc-4138-8f05-ba118fd32e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d"
Jul 17 10:16:20.839454 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:20.839412    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7918923-bcfc-4138-8f05-ba118fd32e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d\" (UID: \"b7918923-bcfc-4138-8f05-ba118fd32e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d"
Jul 17 10:16:20.840091 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:20.840070    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7918923-bcfc-4138-8f05-ba118fd32e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d\" (UID: \"b7918923-bcfc-4138-8f05-ba118fd32e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d"
Jul 17 10:16:21.040256 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:21.040173    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d"
Jul 17 10:16:21.186796 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:21.185313    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d"]
Jul 17 10:16:21.569541 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:21.569499    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d" event={"ID":"b7918923-bcfc-4138-8f05-ba118fd32e31","Type":"ContainerStarted","Data":"4cd6ae48206dc68735ad1f9325b9327d43978520cbf4a472b76b6fd167c15742"}
Jul 17 10:16:21.569686 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:21.569570    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d" event={"ID":"b7918923-bcfc-4138-8f05-ba118fd32e31","Type":"ContainerStarted","Data":"fcfbd9587bba4f40b4e438e07f099d9a878ece2aeb1e1ac89a823f009ed6248e"}
Jul 17 10:16:21.592139 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:21.592094    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d" podStartSLOduration=1.592081616 podStartE2EDuration="1.592081616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:16:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:16:21.591322367 +0000 UTC m=+3465.656143090" watchObservedRunningTime="2026-07-17 10:16:21.592081616 +0000 UTC m=+3465.656902337"
Jul 17 10:16:22.577205 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:22.577165    2571 generic.go:356] "Generic (PLEG): container finished" podID="b7918923-bcfc-4138-8f05-ba118fd32e31" containerID="4cd6ae48206dc68735ad1f9325b9327d43978520cbf4a472b76b6fd167c15742" exitCode=0
Jul 17 10:16:22.577689 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:22.577255    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d" event={"ID":"b7918923-bcfc-4138-8f05-ba118fd32e31","Type":"ContainerDied","Data":"4cd6ae48206dc68735ad1f9325b9327d43978520cbf4a472b76b6fd167c15742"}
Jul 17 10:16:23.733888 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:23.733860    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d"
Jul 17 10:16:23.770457 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:23.770430    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7918923-bcfc-4138-8f05-ba118fd32e31-maas-api-service-ca\") pod \"b7918923-bcfc-4138-8f05-ba118fd32e31\" (UID: \"b7918923-bcfc-4138-8f05-ba118fd32e31\") "
Jul 17 10:16:23.770865 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:23.770837    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7918923-bcfc-4138-8f05-ba118fd32e31-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b7918923-bcfc-4138-8f05-ba118fd32e31" (UID: "b7918923-bcfc-4138-8f05-ba118fd32e31"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:23.871838 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:23.871748    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7918923-bcfc-4138-8f05-ba118fd32e31-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:16:24.591939 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:24.591906    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d"
Jul 17 10:16:24.591939 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:24.591917    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d" event={"ID":"b7918923-bcfc-4138-8f05-ba118fd32e31","Type":"ContainerDied","Data":"fcfbd9587bba4f40b4e438e07f099d9a878ece2aeb1e1ac89a823f009ed6248e"}
Jul 17 10:16:24.592155 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:24.591965    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fcfbd9587bba4f40b4e438e07f099d9a878ece2aeb1e1ac89a823f009ed6248e"
Jul 17 10:16:25.059423 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.059379    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"]
Jul 17 10:16:25.060071 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.060049    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7918923-bcfc-4138-8f05-ba118fd32e31" containerName="revoke-keys"
Jul 17 10:16:25.060168 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.060074    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7918923-bcfc-4138-8f05-ba118fd32e31" containerName="revoke-keys"
Jul 17 10:16:25.060228 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.060211    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7918923-bcfc-4138-8f05-ba118fd32e31" containerName="revoke-keys"
Jul 17 10:16:25.063298 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.063277    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"
Jul 17 10:16:25.071542 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.071511    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"]
Jul 17 10:16:25.082492 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.082433    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa565fbf-d89b-40ba-8a08-5826a0e47c06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g\" (UID: \"aa565fbf-d89b-40ba-8a08-5826a0e47c06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"
Jul 17 10:16:25.184040 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.184002    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa565fbf-d89b-40ba-8a08-5826a0e47c06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g\" (UID: \"aa565fbf-d89b-40ba-8a08-5826a0e47c06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"
Jul 17 10:16:25.184771 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.184750    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa565fbf-d89b-40ba-8a08-5826a0e47c06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g\" (UID: \"aa565fbf-d89b-40ba-8a08-5826a0e47c06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"
Jul 17 10:16:25.377315 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.377222    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"
Jul 17 10:16:25.523916 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.523883    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"]
Jul 17 10:16:25.525342 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:16:25.525312    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa565fbf_d89b_40ba_8a08_5826a0e47c06.slice/crio-734f00bfd1042829e19500643f24e7416c581cb28e4b78217df7e2ff9c74f753 WatchSource:0}: Error finding container 734f00bfd1042829e19500643f24e7416c581cb28e4b78217df7e2ff9c74f753: Status 404 returned error can't find the container with id 734f00bfd1042829e19500643f24e7416c581cb28e4b78217df7e2ff9c74f753
Jul 17 10:16:25.599398 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:25.599358    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g" event={"ID":"aa565fbf-d89b-40ba-8a08-5826a0e47c06","Type":"ContainerStarted","Data":"734f00bfd1042829e19500643f24e7416c581cb28e4b78217df7e2ff9c74f753"}
Jul 17 10:16:26.181776 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:26.181694    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"]
Jul 17 10:16:26.185453 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:26.185436    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"
Jul 17 10:16:26.196151 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:26.196078    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"]
Jul 17 10:16:26.295581 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:26.295546    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d3b5be-07d3-4d8c-847e-baac59fdc53a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879\" (UID: \"e6d3b5be-07d3-4d8c-847e-baac59fdc53a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"
Jul 17 10:16:26.396842 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:26.396803    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d3b5be-07d3-4d8c-847e-baac59fdc53a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879\" (UID: \"e6d3b5be-07d3-4d8c-847e-baac59fdc53a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"
Jul 17 10:16:26.397504 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:26.397450    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d3b5be-07d3-4d8c-847e-baac59fdc53a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879\" (UID: \"e6d3b5be-07d3-4d8c-847e-baac59fdc53a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"
Jul 17 10:16:26.497495 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:26.497427    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"
Jul 17 10:16:26.620709 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:26.620669    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerID="9f6b059db07c3220c079c1b5d609aec470fa6305a35277a0a9442b69d27dca29" exitCode=6
Jul 17 10:16:26.620892 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:26.620779    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g" event={"ID":"aa565fbf-d89b-40ba-8a08-5826a0e47c06","Type":"ContainerDied","Data":"9f6b059db07c3220c079c1b5d609aec470fa6305a35277a0a9442b69d27dca29"}
Jul 17 10:16:26.621064 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:26.621043    2571 scope.go:117] "RemoveContainer" containerID="9f6b059db07c3220c079c1b5d609aec470fa6305a35277a0a9442b69d27dca29"
Jul 17 10:16:26.645849 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:16:26.645817    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d3b5be_07d3_4d8c_847e_baac59fdc53a.slice/crio-a985817bc62d72aedce450cdee0c42ccb5fa06a1f3cb185def535e4e47a0d5b6 WatchSource:0}: Error finding container a985817bc62d72aedce450cdee0c42ccb5fa06a1f3cb185def535e4e47a0d5b6: Status 404 returned error can't find the container with id a985817bc62d72aedce450cdee0c42ccb5fa06a1f3cb185def535e4e47a0d5b6
Jul 17 10:16:26.647605 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:26.647585    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"]
Jul 17 10:16:27.629373 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:27.629333    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerID="9c13745889e3548105d2910968d2bd6d22aace31b7e0f77ef7b3fff5dfea6968" exitCode=6
Jul 17 10:16:27.629854 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:27.629419    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g" event={"ID":"aa565fbf-d89b-40ba-8a08-5826a0e47c06","Type":"ContainerDied","Data":"9c13745889e3548105d2910968d2bd6d22aace31b7e0f77ef7b3fff5dfea6968"}
Jul 17 10:16:27.629854 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:27.629508    2571 scope.go:117] "RemoveContainer" containerID="9f6b059db07c3220c079c1b5d609aec470fa6305a35277a0a9442b69d27dca29"
Jul 17 10:16:27.629854 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:27.629760    2571 scope.go:117] "RemoveContainer" containerID="9c13745889e3548105d2910968d2bd6d22aace31b7e0f77ef7b3fff5dfea6968"
Jul 17 10:16:27.630043 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:16:27.630021    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g_odh-ai-gateway-infra(aa565fbf-d89b-40ba-8a08-5826a0e47c06)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06"
Jul 17 10:16:27.631215 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:27.631193    2571 generic.go:356] "Generic (PLEG): container finished" podID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerID="ba5b289fa9c91df202962a39e905b4c5cc00740107505d3aef5f5ef0c35baa93" exitCode=6
Jul 17 10:16:27.631290 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:27.631245    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879" event={"ID":"e6d3b5be-07d3-4d8c-847e-baac59fdc53a","Type":"ContainerDied","Data":"ba5b289fa9c91df202962a39e905b4c5cc00740107505d3aef5f5ef0c35baa93"}
Jul 17 10:16:27.631332 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:27.631299    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879" event={"ID":"e6d3b5be-07d3-4d8c-847e-baac59fdc53a","Type":"ContainerStarted","Data":"a985817bc62d72aedce450cdee0c42ccb5fa06a1f3cb185def535e4e47a0d5b6"}
Jul 17 10:16:27.631457 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:27.631423    2571 scope.go:117] "RemoveContainer" containerID="ba5b289fa9c91df202962a39e905b4c5cc00740107505d3aef5f5ef0c35baa93"
Jul 17 10:16:28.640439 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:28.640358    2571 scope.go:117] "RemoveContainer" containerID="9c13745889e3548105d2910968d2bd6d22aace31b7e0f77ef7b3fff5dfea6968"
Jul 17 10:16:28.640895 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:16:28.640626    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g_odh-ai-gateway-infra(aa565fbf-d89b-40ba-8a08-5826a0e47c06)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06"
Jul 17 10:16:28.642332 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:28.642309    2571 generic.go:356] "Generic (PLEG): container finished" podID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerID="6494f1771ac826d75dbb1869b26d4a296f8b1fe51c3a4a55811fb135e5ae37a9" exitCode=6
Jul 17 10:16:28.642481 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:28.642395    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879" event={"ID":"e6d3b5be-07d3-4d8c-847e-baac59fdc53a","Type":"ContainerDied","Data":"6494f1771ac826d75dbb1869b26d4a296f8b1fe51c3a4a55811fb135e5ae37a9"}
Jul 17 10:16:28.642481 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:28.642453    2571 scope.go:117] "RemoveContainer" containerID="ba5b289fa9c91df202962a39e905b4c5cc00740107505d3aef5f5ef0c35baa93"
Jul 17 10:16:28.642604 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:28.642586    2571 scope.go:117] "RemoveContainer" containerID="6494f1771ac826d75dbb1869b26d4a296f8b1fe51c3a4a55811fb135e5ae37a9"
Jul 17 10:16:28.642838 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:16:28.642819    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879_odh-ai-gateway-infra(e6d3b5be-07d3-4d8c-847e-baac59fdc53a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a"
Jul 17 10:16:28.752691 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:28.752661    2571 scope.go:117] "RemoveContainer" containerID="68f9c4454e21fab2223c9f4e727a8f83849a26308007a9cdfbe66617e4a9fd06"
Jul 17 10:16:29.651549 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:29.651516    2571 scope.go:117] "RemoveContainer" containerID="6494f1771ac826d75dbb1869b26d4a296f8b1fe51c3a4a55811fb135e5ae37a9"
Jul 17 10:16:29.651995 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:16:29.651746    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879_odh-ai-gateway-infra(e6d3b5be-07d3-4d8c-847e-baac59fdc53a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a"
Jul 17 10:16:29.653309 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:29.653287    2571 generic.go:356] "Generic (PLEG): container finished" podID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerID="7b4597086216586207fded885372ed8e2b154d551fab34800d4cfac29d12ad97" exitCode=6
Jul 17 10:16:29.653443 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:29.653365    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp" event={"ID":"94e0cf34-9d7e-47d1-8e6c-ae890251b456","Type":"ContainerDied","Data":"7b4597086216586207fded885372ed8e2b154d551fab34800d4cfac29d12ad97"}
Jul 17 10:16:29.653443 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:29.653436    2571 scope.go:117] "RemoveContainer" containerID="68f9c4454e21fab2223c9f4e727a8f83849a26308007a9cdfbe66617e4a9fd06"
Jul 17 10:16:29.653728 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:29.653713    2571 scope.go:117] "RemoveContainer" containerID="7b4597086216586207fded885372ed8e2b154d551fab34800d4cfac29d12ad97"
Jul 17 10:16:29.653938 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:16:29.653917    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp_odh-ai-gateway-infra(94e0cf34-9d7e-47d1-8e6c-ae890251b456)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456"
Jul 17 10:16:30.732427 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:30.732117    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"]
Jul 17 10:16:30.882753 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:30.882725    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"
Jul 17 10:16:30.942207 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:30.942175    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94e0cf34-9d7e-47d1-8e6c-ae890251b456-maas-api-service-ca\") pod \"94e0cf34-9d7e-47d1-8e6c-ae890251b456\" (UID: \"94e0cf34-9d7e-47d1-8e6c-ae890251b456\") "
Jul 17 10:16:30.942576 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:30.942553    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94e0cf34-9d7e-47d1-8e6c-ae890251b456-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "94e0cf34-9d7e-47d1-8e6c-ae890251b456" (UID: "94e0cf34-9d7e-47d1-8e6c-ae890251b456"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:31.042933 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:31.042852    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94e0cf34-9d7e-47d1-8e6c-ae890251b456-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:16:31.464397 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:31.464357    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45"]
Jul 17 10:16:31.467150 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:31.467127    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-8d12ab-29738055-nhn45"]
Jul 17 10:16:31.669438 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:31.669408    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"
Jul 17 10:16:31.669606 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:31.669447    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp" event={"ID":"94e0cf34-9d7e-47d1-8e6c-ae890251b456","Type":"ContainerDied","Data":"889211cea9a231225e5f7250d71ab95fa44c002942371390a70daf08b79231a2"}
Jul 17 10:16:31.669606 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:31.669525    2571 scope.go:117] "RemoveContainer" containerID="7b4597086216586207fded885372ed8e2b154d551fab34800d4cfac29d12ad97"
Jul 17 10:16:31.698370 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:31.698315    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"]
Jul 17 10:16:31.707846 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:31.707810    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5593c7b6-6r7rp"]
Jul 17 10:16:31.758955 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:31.758863    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a72cc1a-6f3e-4ea5-83d9-b8e18555d5f9" path="/var/lib/kubelet/pods/8a72cc1a-6f3e-4ea5-83d9-b8e18555d5f9/volumes"
Jul 17 10:16:31.759408 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:31.759293    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" path="/var/lib/kubelet/pods/94e0cf34-9d7e-47d1-8e6c-ae890251b456/volumes"
Jul 17 10:16:39.752580 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:39.752547    2571 scope.go:117] "RemoveContainer" containerID="9c13745889e3548105d2910968d2bd6d22aace31b7e0f77ef7b3fff5dfea6968"
Jul 17 10:16:40.726452 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:40.726420    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerID="5ee58ad843b9b44687b7d5475ebbb5ddcc53d3772e1ec47fc5f4cb3f996d6a5a" exitCode=6
Jul 17 10:16:40.726625 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:40.726496    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g" event={"ID":"aa565fbf-d89b-40ba-8a08-5826a0e47c06","Type":"ContainerDied","Data":"5ee58ad843b9b44687b7d5475ebbb5ddcc53d3772e1ec47fc5f4cb3f996d6a5a"}
Jul 17 10:16:40.726625 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:40.726559    2571 scope.go:117] "RemoveContainer" containerID="9c13745889e3548105d2910968d2bd6d22aace31b7e0f77ef7b3fff5dfea6968"
Jul 17 10:16:40.726933 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:40.726909    2571 scope.go:117] "RemoveContainer" containerID="5ee58ad843b9b44687b7d5475ebbb5ddcc53d3772e1ec47fc5f4cb3f996d6a5a"
Jul 17 10:16:40.727190 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:16:40.727162    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g_odh-ai-gateway-infra(aa565fbf-d89b-40ba-8a08-5826a0e47c06)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06"
Jul 17 10:16:40.752526 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:40.752494    2571 scope.go:117] "RemoveContainer" containerID="6494f1771ac826d75dbb1869b26d4a296f8b1fe51c3a4a55811fb135e5ae37a9"
Jul 17 10:16:41.736426 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:41.736398    2571 generic.go:356] "Generic (PLEG): container finished" podID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerID="512319c5f18d60ee97fe9658fc62ceaa6b096371664831e8257f37d94073a761" exitCode=6
Jul 17 10:16:41.736826 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:41.736488    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879" event={"ID":"e6d3b5be-07d3-4d8c-847e-baac59fdc53a","Type":"ContainerDied","Data":"512319c5f18d60ee97fe9658fc62ceaa6b096371664831e8257f37d94073a761"}
Jul 17 10:16:41.736826 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:41.736539    2571 scope.go:117] "RemoveContainer" containerID="6494f1771ac826d75dbb1869b26d4a296f8b1fe51c3a4a55811fb135e5ae37a9"
Jul 17 10:16:41.736935 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:41.736882    2571 scope.go:117] "RemoveContainer" containerID="512319c5f18d60ee97fe9658fc62ceaa6b096371664831e8257f37d94073a761"
Jul 17 10:16:41.737094 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:16:41.737073    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879_odh-ai-gateway-infra(e6d3b5be-07d3-4d8c-847e-baac59fdc53a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a"
Jul 17 10:16:41.769385 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:41.769347    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"]
Jul 17 10:16:41.916565 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:41.916536    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"
Jul 17 10:16:42.044666 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:42.044568    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa565fbf-d89b-40ba-8a08-5826a0e47c06-maas-api-service-ca\") pod \"aa565fbf-d89b-40ba-8a08-5826a0e47c06\" (UID: \"aa565fbf-d89b-40ba-8a08-5826a0e47c06\") "
Jul 17 10:16:42.044957 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:42.044932    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa565fbf-d89b-40ba-8a08-5826a0e47c06-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aa565fbf-d89b-40ba-8a08-5826a0e47c06" (UID: "aa565fbf-d89b-40ba-8a08-5826a0e47c06"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:42.145727 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:42.145690    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa565fbf-d89b-40ba-8a08-5826a0e47c06-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:16:42.745830 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:42.745801    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"
Jul 17 10:16:42.746359 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:42.745801    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g" event={"ID":"aa565fbf-d89b-40ba-8a08-5826a0e47c06","Type":"ContainerDied","Data":"734f00bfd1042829e19500643f24e7416c581cb28e4b78217df7e2ff9c74f753"}
Jul 17 10:16:42.746359 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:42.745936    2571 scope.go:117] "RemoveContainer" containerID="5ee58ad843b9b44687b7d5475ebbb5ddcc53d3772e1ec47fc5f4cb3f996d6a5a"
Jul 17 10:16:42.771238 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:42.771199    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"]
Jul 17 10:16:42.774051 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:42.774021    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"]
Jul 17 10:16:42.778831 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:42.778803    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-q4s4g"]
Jul 17 10:16:42.917629 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:42.917604    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"
Jul 17 10:16:43.053784 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:43.053688    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d3b5be-07d3-4d8c-847e-baac59fdc53a-maas-api-service-ca\") pod \"e6d3b5be-07d3-4d8c-847e-baac59fdc53a\" (UID: \"e6d3b5be-07d3-4d8c-847e-baac59fdc53a\") "
Jul 17 10:16:43.054056 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:43.054033    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6d3b5be-07d3-4d8c-847e-baac59fdc53a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6d3b5be-07d3-4d8c-847e-baac59fdc53a" (UID: "e6d3b5be-07d3-4d8c-847e-baac59fdc53a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:43.154606 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:43.154550    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6d3b5be-07d3-4d8c-847e-baac59fdc53a-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:16:43.753762 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:43.753726    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"
Jul 17 10:16:43.760627 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:43.760599    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" path="/var/lib/kubelet/pods/aa565fbf-d89b-40ba-8a08-5826a0e47c06/volumes"
Jul 17 10:16:43.761082 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:43.761059    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879" event={"ID":"e6d3b5be-07d3-4d8c-847e-baac59fdc53a","Type":"ContainerDied","Data":"a985817bc62d72aedce450cdee0c42ccb5fa06a1f3cb185def535e4e47a0d5b6"}
Jul 17 10:16:43.761153 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:43.761125    2571 scope.go:117] "RemoveContainer" containerID="512319c5f18d60ee97fe9658fc62ceaa6b096371664831e8257f37d94073a761"
Jul 17 10:16:43.797161 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:43.797116    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"]
Jul 17 10:16:43.809882 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:43.809843    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-8f9d7266-2j879"]
Jul 17 10:16:45.759710 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:16:45.759676    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" path="/var/lib/kubelet/pods/e6d3b5be-07d3-4d8c-847e-baac59fdc53a/volumes"
Jul 17 10:17:04.114257 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:04.114182    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-j6d95_e9763398-f47f-4a5e-9048-776d3b931e4f/manager/0.log"
Jul 17 10:17:04.232445 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:04.232410    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mh7dz_9451d4ab-821c-4361-8f98-963d0f02f5b1/manager/0.log"
Jul 17 10:17:04.360745 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:04.360707    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-769d7b9cbc-6nlfc_029c86fc-87ae-42fb-b508-9c967032c8cb/manager/0.log"
Jul 17 10:17:04.476025 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:04.475994    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-g6z4w_aa670365-d71a-447d-98aa-74077e385b07/server/0.log"
Jul 17 10:17:04.600773 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:04.600725    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jgsgk_336327ee-2ed2-4751-8516-b8d54a64900f/manager/0.log"
Jul 17 10:17:04.711608 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:04.711579    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdbb8988-fhn4b_b715bf43-21ac-4495-9a18-fb596c629ab0/copy-manifests/0.log"
Jul 17 10:17:04.719552 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:04.719501    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdbb8988-fhn4b_b715bf43-21ac-4495-9a18-fb596c629ab0/manager/0.log"
Jul 17 10:17:05.336953 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:05.336915    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-57ddf56f55-kj8cc_e727bc87-1596-4a13-b51a-72d3323e2886/maas-api/0.log"
Jul 17 10:17:05.653736 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:05.653660    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738055-52gvn_fa687682-dbf6-4234-aa37-895de3ebf353/cleanup/0.log"
Jul 17 10:17:05.868724 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:05.868689    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-d7262f-x24mz_fdc51fb3-ae26-4fa3-b820-f5a3aca8b07b/revoke-keys/0.log"
Jul 17 10:17:06.209519 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:06.209491    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-fe50f0-ppzhb_00699b1c-f31f-4635-8427-353fecff117f/revoke-keys/0.log"
Jul 17 10:17:06.320558 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:06.320517    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-787694-9ktj9_f300379f-7ce6-48f2-86ed-98bb66d2ca79/revoke-keys/0.log"
Jul 17 10:17:06.427602 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:06.427575    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-0d3f0022-pg4bd_1b12cee4-ac8a-4c89-84a7-a05bfe9528c8/revoke-keys/0.log"
Jul 17 10:17:06.535874 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:06.535795    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-43c677b2-fznxs_bb555f53-125f-41ac-8e46-c4ce4bf40eb9/revoke-keys/0.log"
Jul 17 10:17:06.641096 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:06.641069    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-8ae1eb-nf786_51f2b841-3aa2-4bd3-b0b1-01108304b63a/revoke-keys/0.log"
Jul 17 10:17:06.748581 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:06.748550    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-8d12ab-scc2d_b7918923-bcfc-4138-8f05-ba118fd32e31/revoke-keys/0.log"
Jul 17 10:17:06.856138 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:06.856051    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-x59b8_b585bb07-1d6e-4cff-bb0b-7b77d91f3e73/postgres/0.log"
Jul 17 10:17:07.604018 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.603978    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq_c22918ea-3e5c-4cf0-997a-60924541367c/pull/0.log"
Jul 17 10:17:07.610177 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.610140    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq_c22918ea-3e5c-4cf0-997a-60924541367c/extract/0.log"
Jul 17 10:17:07.616109 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.616090    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq_c22918ea-3e5c-4cf0-997a-60924541367c/util/0.log"
Jul 17 10:17:07.721179 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.721149    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf_a853521b-dc4b-458c-8259-2d4bc3692f97/extract/0.log"
Jul 17 10:17:07.727172 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.727146    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf_a853521b-dc4b-458c-8259-2d4bc3692f97/util/0.log"
Jul 17 10:17:07.733014 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.732991    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf_a853521b-dc4b-458c-8259-2d4bc3692f97/pull/0.log"
Jul 17 10:17:07.834416 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.834384    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2_352624fd-81f2-430b-9ba9-a413d64c108a/util/0.log"
Jul 17 10:17:07.840886 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.840861    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2_352624fd-81f2-430b-9ba9-a413d64c108a/pull/0.log"
Jul 17 10:17:07.848989 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.848959    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2_352624fd-81f2-430b-9ba9-a413d64c108a/extract/0.log"
Jul 17 10:17:07.981004 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.980972    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn_5e8308e7-7b0c-4b8c-988e-b4ffc7a61416/util/0.log"
Jul 17 10:17:07.987185 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.987164    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn_5e8308e7-7b0c-4b8c-988e-b4ffc7a61416/pull/0.log"
Jul 17 10:17:07.992599 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:07.992578    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn_5e8308e7-7b0c-4b8c-988e-b4ffc7a61416/extract/0.log"
Jul 17 10:17:08.223586 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:08.223544    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hdxzl_1012a826-2059-4be3-98a9-eec703236571/manager/0.log"
Jul 17 10:17:08.334142 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:08.334058    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-kdtdq_88845c50-3a6f-4983-bf5d-0bd429c7cac9/manager/0.log"
Jul 17 10:17:09.356205 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:09.356175    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ktvxn_04aa59a4-2be5-498f-b3e6-6c1d9bf92551/discovery/0.log"
Jul 17 10:17:09.465271 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:09.465238    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df6bb5dc-nn7bl_83d184a8-32f9-4b5e-b244-06430e33b380/kube-auth-proxy/0.log"
Jul 17 10:17:09.807701 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:09.807664    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xth9s_505c6160-56a3-4353-973f-b460a979e97a/payload-pre-processing/0.log"
Jul 17 10:17:09.926169 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:09.926135    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lh5rz_12ac5b6c-43fb-44fa-bbca-077b309bd8b8/payload-processing/0.log"
Jul 17 10:17:10.486055 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:10.486014    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-fw775_bf5dcf60-6004-4056-96dd-d2f22070a73f/main/0.log"
Jul 17 10:17:10.825768 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:10.825689    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-bs66m_2dac0dea-f7b5-47d6-a142-59c1182a313e/main/0.log"
Jul 17 10:17:15.110844 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.110805    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"]
Jul 17 10:17:15.111305 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111265    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerName="revoke-keys"
Jul 17 10:17:15.111305 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111278    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerName="revoke-keys"
Jul 17 10:17:15.111305 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111289    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerName="revoke-keys"
Jul 17 10:17:15.111305 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111296    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111307    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111313    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111324    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111329    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111335    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111341    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111349    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111355    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111364    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111369    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111433    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111441    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerName="revoke-keys"
Jul 17 10:17:15.111442 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111447    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerName="revoke-keys"
Jul 17 10:17:15.111895 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111454    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerName="revoke-keys"
Jul 17 10:17:15.111895 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111474    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerName="revoke-keys"
Jul 17 10:17:15.111895 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.111481    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerName="revoke-keys"
Jul 17 10:17:15.114603 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.114586    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"
Jul 17 10:17:15.117744 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.117724    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8lndl\""
Jul 17 10:17:15.123605 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.123580    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"]
Jul 17 10:17:15.147975 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.147932    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b146f642-e592-4cfb-ad31-0209448091ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg\" (UID: \"b146f642-e592-4cfb-ad31-0209448091ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"
Jul 17 10:17:15.248830 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.248787    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b146f642-e592-4cfb-ad31-0209448091ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg\" (UID: \"b146f642-e592-4cfb-ad31-0209448091ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"
Jul 17 10:17:15.249410 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.249382    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b146f642-e592-4cfb-ad31-0209448091ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg\" (UID: \"b146f642-e592-4cfb-ad31-0209448091ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"
Jul 17 10:17:15.427876 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.427777    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"
Jul 17 10:17:15.570319 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.570279    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"]
Jul 17 10:17:15.571723 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:17:15.571684    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb146f642_e592_4cfb_ad31_0209448091ff.slice/crio-6dd2ce3ef39dfedab47a94aed071c8a0e0a300151c1da07646ca22da5e63e6e9 WatchSource:0}: Error finding container 6dd2ce3ef39dfedab47a94aed071c8a0e0a300151c1da07646ca22da5e63e6e9: Status 404 returned error can't find the container with id 6dd2ce3ef39dfedab47a94aed071c8a0e0a300151c1da07646ca22da5e63e6e9
Jul 17 10:17:15.958924 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.958872    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg" event={"ID":"b146f642-e592-4cfb-ad31-0209448091ff","Type":"ContainerStarted","Data":"e2ab8962d951110b282ec92ad515e9b978fd4df56f01a60226384d51de99b864"}
Jul 17 10:17:15.958924 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.958929    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg" event={"ID":"b146f642-e592-4cfb-ad31-0209448091ff","Type":"ContainerStarted","Data":"6dd2ce3ef39dfedab47a94aed071c8a0e0a300151c1da07646ca22da5e63e6e9"}
Jul 17 10:17:15.977967 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:15.977907    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg" podStartSLOduration=0.977890466 podStartE2EDuration="977.890466ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:17:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:17:15.977060742 +0000 UTC m=+3520.041881465" watchObservedRunningTime="2026-07-17 10:17:15.977890466 +0000 UTC m=+3520.042711187"
Jul 17 10:17:16.966393 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:16.966361    2571 generic.go:356] "Generic (PLEG): container finished" podID="b146f642-e592-4cfb-ad31-0209448091ff" containerID="e2ab8962d951110b282ec92ad515e9b978fd4df56f01a60226384d51de99b864" exitCode=6
Jul 17 10:17:16.966831 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:16.966448    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg" event={"ID":"b146f642-e592-4cfb-ad31-0209448091ff","Type":"ContainerDied","Data":"e2ab8962d951110b282ec92ad515e9b978fd4df56f01a60226384d51de99b864"}
Jul 17 10:17:16.966831 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:16.966661    2571 scope.go:117] "RemoveContainer" containerID="e2ab8962d951110b282ec92ad515e9b978fd4df56f01a60226384d51de99b864"
Jul 17 10:17:17.975435 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:17.975406    2571 generic.go:356] "Generic (PLEG): container finished" podID="b146f642-e592-4cfb-ad31-0209448091ff" containerID="d1183f467d0aa14cd52f27f35bba42b4dc78aa8ecf00823e8cdae13d1a1f0bca" exitCode=6
Jul 17 10:17:17.975899 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:17.975446    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg" event={"ID":"b146f642-e592-4cfb-ad31-0209448091ff","Type":"ContainerDied","Data":"d1183f467d0aa14cd52f27f35bba42b4dc78aa8ecf00823e8cdae13d1a1f0bca"}
Jul 17 10:17:17.975899 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:17.975514    2571 scope.go:117] "RemoveContainer" containerID="e2ab8962d951110b282ec92ad515e9b978fd4df56f01a60226384d51de99b864"
Jul 17 10:17:17.975899 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:17.975704    2571 scope.go:117] "RemoveContainer" containerID="d1183f467d0aa14cd52f27f35bba42b4dc78aa8ecf00823e8cdae13d1a1f0bca"
Jul 17 10:17:17.976029 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:17:17.975931    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg_odh-ai-gateway-infra(b146f642-e592-4cfb-ad31-0209448091ff)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg" podUID="b146f642-e592-4cfb-ad31-0209448091ff"
Jul 17 10:17:18.982982 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:18.982941    2571 scope.go:117] "RemoveContainer" containerID="d1183f467d0aa14cd52f27f35bba42b4dc78aa8ecf00823e8cdae13d1a1f0bca"
Jul 17 10:17:18.983407 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:17:18.983131    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg_odh-ai-gateway-infra(b146f642-e592-4cfb-ad31-0209448091ff)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg" podUID="b146f642-e592-4cfb-ad31-0209448091ff"
Jul 17 10:17:20.794786 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:20.794752    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5tsrt_0970881d-9524-4242-bc47-7f0813bf2df1/global-pull-secret-syncer/0.log"
Jul 17 10:17:20.986639 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:20.986605    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hhg49_5c0e214a-055e-4903-864b-7a86e3cc05c5/konnectivity-agent/0.log"
Jul 17 10:17:21.082053 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:21.081970    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-208.ec2.internal_b1ab77a29ff5b283c6ddae565b57e84f/haproxy/0.log"
Jul 17 10:17:26.034700 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.034666    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq_c22918ea-3e5c-4cf0-997a-60924541367c/extract/0.log"
Jul 17 10:17:26.063254 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.063212    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq_c22918ea-3e5c-4cf0-997a-60924541367c/util/0.log"
Jul 17 10:17:26.106937 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.106908    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pflxq_c22918ea-3e5c-4cf0-997a-60924541367c/pull/0.log"
Jul 17 10:17:26.155719 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.155687    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf_a853521b-dc4b-458c-8259-2d4bc3692f97/extract/0.log"
Jul 17 10:17:26.193891 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.193853    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf_a853521b-dc4b-458c-8259-2d4bc3692f97/util/0.log"
Jul 17 10:17:26.224222 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.224195    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06l9cf_a853521b-dc4b-458c-8259-2d4bc3692f97/pull/0.log"
Jul 17 10:17:26.262527 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.262493    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2_352624fd-81f2-430b-9ba9-a413d64c108a/extract/0.log"
Jul 17 10:17:26.299777 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.299687    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2_352624fd-81f2-430b-9ba9-a413d64c108a/util/0.log"
Jul 17 10:17:26.331062 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.331032    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739j6s2_352624fd-81f2-430b-9ba9-a413d64c108a/pull/0.log"
Jul 17 10:17:26.370076 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.370051    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn_5e8308e7-7b0c-4b8c-988e-b4ffc7a61416/extract/0.log"
Jul 17 10:17:26.393504 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.393454    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn_5e8308e7-7b0c-4b8c-988e-b4ffc7a61416/util/0.log"
Jul 17 10:17:26.422528 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.422494    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16lshn_5e8308e7-7b0c-4b8c-988e-b4ffc7a61416/pull/0.log"
Jul 17 10:17:26.788736 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.788698    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hdxzl_1012a826-2059-4be3-98a9-eec703236571/manager/0.log"
Jul 17 10:17:26.820259 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:26.820227    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-kdtdq_88845c50-3a6f-4983-bf5d-0bd429c7cac9/manager/0.log"
Jul 17 10:17:28.855851 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:28.855781    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-jprcn_da993dbb-9c00-47e6-a410-8c8692e4644c/cluster-monitoring-operator/0.log"
Jul 17 10:17:28.889048 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:28.889010    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-zsf4n_10c073d2-4eee-45f2-9f5e-e15b7afc16d5/kube-state-metrics/0.log"
Jul 17 10:17:28.909340 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:28.909311    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-zsf4n_10c073d2-4eee-45f2-9f5e-e15b7afc16d5/kube-rbac-proxy-main/0.log"
Jul 17 10:17:28.933518 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:28.933488    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-zsf4n_10c073d2-4eee-45f2-9f5e-e15b7afc16d5/kube-rbac-proxy-self/0.log"
Jul 17 10:17:28.991731 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:28.991706    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-nn2z7_6eeeaefe-7922-46ee-a6f4-9d1fd4268fc7/monitoring-plugin/0.log"
Jul 17 10:17:29.024090 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:29.024058    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sp4v_25dfceba-0dab-4034-8441-364ad8a10ba1/node-exporter/0.log"
Jul 17 10:17:29.046546 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:29.046513    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sp4v_25dfceba-0dab-4034-8441-364ad8a10ba1/kube-rbac-proxy/0.log"
Jul 17 10:17:29.069265 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:29.069238    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sp4v_25dfceba-0dab-4034-8441-364ad8a10ba1/init-textfile/0.log"
Jul 17 10:17:29.621153 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:29.621120    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-l8zfg_f0ca7342-a808-4f5a-a119-9efb356b05f8/prometheus-operator-admission-webhook/0.log"
Jul 17 10:17:29.730612 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:29.730585    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59d865899b-9pxjh_8cb71f1a-71a5-4302-8751-f2ec2196b256/thanos-query/0.log"
Jul 17 10:17:29.753036 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:29.753009    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59d865899b-9pxjh_8cb71f1a-71a5-4302-8751-f2ec2196b256/kube-rbac-proxy-web/0.log"
Jul 17 10:17:29.776379 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:29.776342    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59d865899b-9pxjh_8cb71f1a-71a5-4302-8751-f2ec2196b256/kube-rbac-proxy/0.log"
Jul 17 10:17:29.808197 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:29.808167    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59d865899b-9pxjh_8cb71f1a-71a5-4302-8751-f2ec2196b256/prom-label-proxy/0.log"
Jul 17 10:17:29.832960 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:29.832921    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59d865899b-9pxjh_8cb71f1a-71a5-4302-8751-f2ec2196b256/kube-rbac-proxy-rules/0.log"
Jul 17 10:17:29.859322 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:29.859286    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-59d865899b-9pxjh_8cb71f1a-71a5-4302-8751-f2ec2196b256/kube-rbac-proxy-metrics/0.log"
Jul 17 10:17:30.544207 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.544172    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"]
Jul 17 10:17:30.544636 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.544623    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerName="revoke-keys"
Jul 17 10:17:30.544705 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.544638    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerName="revoke-keys"
Jul 17 10:17:30.544705 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.544657    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerName="revoke-keys"
Jul 17 10:17:30.544705 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.544663    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerName="revoke-keys"
Jul 17 10:17:30.544801 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.544735    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa565fbf-d89b-40ba-8a08-5826a0e47c06" containerName="revoke-keys"
Jul 17 10:17:30.544801 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.544745    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6d3b5be-07d3-4d8c-847e-baac59fdc53a" containerName="revoke-keys"
Jul 17 10:17:30.544801 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.544754    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="94e0cf34-9d7e-47d1-8e6c-ae890251b456" containerName="revoke-keys"
Jul 17 10:17:30.548035 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.548013    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.552647 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.552629    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z5nh8\"/\"default-dockercfg-qnv6m\""
Jul 17 10:17:30.555516 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.555500    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z5nh8\"/\"openshift-service-ca.crt\""
Jul 17 10:17:30.555627 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.555500    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z5nh8\"/\"kube-root-ca.crt\""
Jul 17 10:17:30.573729 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.573700    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"]
Jul 17 10:17:30.691853 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.691820    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-proc\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.692042 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.691956    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-sys\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.692042 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.691991    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-lib-modules\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.692042 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.692010    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-podres\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.692042 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.692030    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw5wc\" (UniqueName: \"kubernetes.io/projected/addd36da-7200-40ce-910a-e63c68936bea-kube-api-access-vw5wc\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.793280 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.793219    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-proc\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.793507 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.793320    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-proc\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.793507 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.793379    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-sys\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.793507 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.793402    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-lib-modules\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.793507 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.793427    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-podres\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.793507 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.793457    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw5wc\" (UniqueName: \"kubernetes.io/projected/addd36da-7200-40ce-910a-e63c68936bea-kube-api-access-vw5wc\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.793507 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.793493    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-sys\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.793846 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.793578    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-lib-modules\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.793846 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.793612    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/addd36da-7200-40ce-910a-e63c68936bea-podres\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.807084 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.807016    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw5wc\" (UniqueName: \"kubernetes.io/projected/addd36da-7200-40ce-910a-e63c68936bea-kube-api-access-vw5wc\") pod \"perf-node-gather-daemonset-9lpsw\" (UID: \"addd36da-7200-40ce-910a-e63c68936bea\") " pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:30.857793 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:30.857754    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:31.007286 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:31.007235    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"]
Jul 17 10:17:31.013846 ip-10-0-138-208 kubenswrapper[2571]: W0717 10:17:31.013819    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaddd36da_7200_40ce_910a_e63c68936bea.slice/crio-4ac3523ee0d012243bc074a52487ab3569f7988a2e23aa8ab204d60c2f5ecbe6 WatchSource:0}: Error finding container 4ac3523ee0d012243bc074a52487ab3569f7988a2e23aa8ab204d60c2f5ecbe6: Status 404 returned error can't find the container with id 4ac3523ee0d012243bc074a52487ab3569f7988a2e23aa8ab204d60c2f5ecbe6
Jul 17 10:17:31.057511 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:31.057434    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw" event={"ID":"addd36da-7200-40ce-910a-e63c68936bea","Type":"ContainerStarted","Data":"4ac3523ee0d012243bc074a52487ab3569f7988a2e23aa8ab204d60c2f5ecbe6"}
Jul 17 10:17:31.086941 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:31.086908    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-9pmr5_7452d789-b8dd-4ccc-b528-84ece1143cf9/networking-console-plugin/0.log"
Jul 17 10:17:31.753062 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:31.753012    2571 scope.go:117] "RemoveContainer" containerID="d1183f467d0aa14cd52f27f35bba42b4dc78aa8ecf00823e8cdae13d1a1f0bca"
Jul 17 10:17:32.066055 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:32.066023    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg" event={"ID":"b146f642-e592-4cfb-ad31-0209448091ff","Type":"ContainerStarted","Data":"e53f1c662357230b86e6d343fa3a23f2de59a81111914b3a0701ae68e7e6f06b"}
Jul 17 10:17:32.067493 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:32.067443    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw" event={"ID":"addd36da-7200-40ce-910a-e63c68936bea","Type":"ContainerStarted","Data":"aaf7db7e33e92e2e4bebeb3bfb32bde402177965c14b86a7b0a91d36bebec4a3"}
Jul 17 10:17:32.067594 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:32.067521    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:32.109209 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:32.109155    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw" podStartSLOduration=2.109137995 podStartE2EDuration="2.109137995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:17:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:17:32.106453454 +0000 UTC m=+3536.171274213" watchObservedRunningTime="2026-07-17 10:17:32.109137995 +0000 UTC m=+3536.173958743"
Jul 17 10:17:32.356655 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:32.356572    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-5fnqc_9b64b23e-d02b-4b7b-9abb-edb2afe4b0a8/download-server/0.log"
Jul 17 10:17:33.042388 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:33.042336    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-nmj78_bcfb2609-29dc-4ae7-ab3a-3a60d2334520/volume-data-source-validator/0.log"
Jul 17 10:17:33.076515 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:33.076485    2571 generic.go:356] "Generic (PLEG): container finished" podID="b146f642-e592-4cfb-ad31-0209448091ff" containerID="e53f1c662357230b86e6d343fa3a23f2de59a81111914b3a0701ae68e7e6f06b" exitCode=6
Jul 17 10:17:33.076930 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:33.076537    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg" event={"ID":"b146f642-e592-4cfb-ad31-0209448091ff","Type":"ContainerDied","Data":"e53f1c662357230b86e6d343fa3a23f2de59a81111914b3a0701ae68e7e6f06b"}
Jul 17 10:17:33.076930 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:33.076599    2571 scope.go:117] "RemoveContainer" containerID="d1183f467d0aa14cd52f27f35bba42b4dc78aa8ecf00823e8cdae13d1a1f0bca"
Jul 17 10:17:33.076930 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:33.076846    2571 scope.go:117] "RemoveContainer" containerID="e53f1c662357230b86e6d343fa3a23f2de59a81111914b3a0701ae68e7e6f06b"
Jul 17 10:17:33.077098 ip-10-0-138-208 kubenswrapper[2571]: E0717 10:17:33.077080    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg_odh-ai-gateway-infra(b146f642-e592-4cfb-ad31-0209448091ff)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg" podUID="b146f642-e592-4cfb-ad31-0209448091ff"
Jul 17 10:17:33.104899 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:33.104871    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"]
Jul 17 10:17:34.227883 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:34.227858    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"
Jul 17 10:17:34.239198 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:34.239170    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pvp8l_4db5b93e-8ebb-4d86-b45a-0bee6a6890f8/dns/0.log"
Jul 17 10:17:34.269975 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:34.269938    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pvp8l_4db5b93e-8ebb-4d86-b45a-0bee6a6890f8/kube-rbac-proxy/0.log"
Jul 17 10:17:34.325742 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:34.325709    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b146f642-e592-4cfb-ad31-0209448091ff-maas-api-service-ca\") pod \"b146f642-e592-4cfb-ad31-0209448091ff\" (UID: \"b146f642-e592-4cfb-ad31-0209448091ff\") "
Jul 17 10:17:34.326094 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:34.326071    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b146f642-e592-4cfb-ad31-0209448091ff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b146f642-e592-4cfb-ad31-0209448091ff" (UID: "b146f642-e592-4cfb-ad31-0209448091ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:17:34.389917 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:34.389838    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n5hmq_199a5035-60c1-4b0d-941e-4a1a20291037/dns-node-resolver/0.log"
Jul 17 10:17:34.426410 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:34.426370    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b146f642-e592-4cfb-ad31-0209448091ff-maas-api-service-ca\") on node \"ip-10-0-138-208.ec2.internal\" DevicePath \"\""
Jul 17 10:17:35.010578 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:35.010547    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-79b9779f5d-fzv2c_37ac236b-cf30-427e-b6a9-0e1ee305a56e/registry/0.log"
Jul 17 10:17:35.070571 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:35.070532    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ksfkn_53816944-d4bc-4e11-8c58-b9427233583a/node-ca/0.log"
Jul 17 10:17:35.091276 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:35.091249    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"
Jul 17 10:17:35.091457 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:35.091293    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg" event={"ID":"b146f642-e592-4cfb-ad31-0209448091ff","Type":"ContainerDied","Data":"6dd2ce3ef39dfedab47a94aed071c8a0e0a300151c1da07646ca22da5e63e6e9"}
Jul 17 10:17:35.091457 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:35.091362    2571 scope.go:117] "RemoveContainer" containerID="e53f1c662357230b86e6d343fa3a23f2de59a81111914b3a0701ae68e7e6f06b"
Jul 17 10:17:35.114857 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:35.114828    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"]
Jul 17 10:17:35.120614 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:35.120585    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-58adf1ce-42wzg"]
Jul 17 10:17:35.758916 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:35.758887    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b146f642-e592-4cfb-ad31-0209448091ff" path="/var/lib/kubelet/pods/b146f642-e592-4cfb-ad31-0209448091ff/volumes"
Jul 17 10:17:36.358075 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:36.358039    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ktvxn_04aa59a4-2be5-498f-b3e6-6c1d9bf92551/discovery/0.log"
Jul 17 10:17:36.379326 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:36.379295    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df6bb5dc-nn7bl_83d184a8-32f9-4b5e-b244-06430e33b380/kube-auth-proxy/0.log"
Jul 17 10:17:36.513947 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:36.513915    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xth9s_505c6160-56a3-4353-973f-b460a979e97a/payload-pre-processing/0.log"
Jul 17 10:17:36.557928 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:36.557895    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lh5rz_12ac5b6c-43fb-44fa-bbca-077b309bd8b8/payload-processing/0.log"
Jul 17 10:17:37.335739 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:37.335706    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l2h6h_b962f3cc-8103-449d-95af-efe0a5d84bbf/serve-healthcheck-canary/0.log"
Jul 17 10:17:37.868359 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:37.868328    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-98q75_96b39859-a448-4d17-aa25-35628ab46939/insights-operator/0.log"
Jul 17 10:17:37.869841 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:37.869813    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-98q75_96b39859-a448-4d17-aa25-35628ab46939/insights-operator/1.log"
Jul 17 10:17:37.889818 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:37.889795    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-48h52_bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb/kube-rbac-proxy/0.log"
Jul 17 10:17:37.913037 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:37.912965    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-48h52_bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb/exporter/0.log"
Jul 17 10:17:37.937658 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:37.937620    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-48h52_bb0b1c0c-9028-4f4f-9ea7-f2da0858bceb/extractor/0.log"
Jul 17 10:17:38.086946 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:38.086919    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z5nh8/perf-node-gather-daemonset-9lpsw"
Jul 17 10:17:40.625339 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:40.625258    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-j6d95_e9763398-f47f-4a5e-9048-776d3b931e4f/manager/0.log"
Jul 17 10:17:40.880191 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:40.880097    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mh7dz_9451d4ab-821c-4361-8f98-963d0f02f5b1/manager/0.log"
Jul 17 10:17:41.059630 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:41.059593    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-769d7b9cbc-6nlfc_029c86fc-87ae-42fb-b508-9c967032c8cb/manager/0.log"
Jul 17 10:17:41.080743 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:41.080705    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-g6z4w_aa670365-d71a-447d-98aa-74077e385b07/server/0.log"
Jul 17 10:17:41.224622 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:41.224586    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jgsgk_336327ee-2ed2-4751-8516-b8d54a64900f/manager/0.log"
Jul 17 10:17:41.254116 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:41.254075    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdbb8988-fhn4b_b715bf43-21ac-4495-9a18-fb596c629ab0/manager/0.log"
Jul 17 10:17:41.281294 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:41.281267    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-76fdbb8988-fhn4b_b715bf43-21ac-4495-9a18-fb596c629ab0/copy-manifests/0.log"
Jul 17 10:17:41.301651 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:41.301621    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ktvxn_04aa59a4-2be5-498f-b3e6-6c1d9bf92551/discovery/0.log"
Jul 17 10:17:41.312868 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:41.312842    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df6bb5dc-nn7bl_83d184a8-32f9-4b5e-b244-06430e33b380/kube-auth-proxy/0.log"
Jul 17 10:17:41.417161 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:41.417129    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xth9s_505c6160-56a3-4353-973f-b460a979e97a/payload-pre-processing/0.log"
Jul 17 10:17:41.448250 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:41.448221    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lh5rz_12ac5b6c-43fb-44fa-bbca-077b309bd8b8/payload-processing/0.log"
Jul 17 10:17:43.240011 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:43.239980    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ktvxn_04aa59a4-2be5-498f-b3e6-6c1d9bf92551/discovery/0.log"
Jul 17 10:17:43.281399 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:43.281371    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df6bb5dc-nn7bl_83d184a8-32f9-4b5e-b244-06430e33b380/kube-auth-proxy/0.log"
Jul 17 10:17:43.385986 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:43.385962    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xth9s_505c6160-56a3-4353-973f-b460a979e97a/payload-pre-processing/0.log"
Jul 17 10:17:43.417201 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:43.417175    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-lh5rz_12ac5b6c-43fb-44fa-bbca-077b309bd8b8/payload-processing/0.log"
Jul 17 10:17:48.261939 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:48.261906    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-hkvkj_debff7eb-0899-41e0-9202-d1dfd5d3a375/migrator/0.log"
Jul 17 10:17:48.312092 ip-10-0-138-208 kubenswrapper[2571]: I0717 10:17:48.312064    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-hkvkj_debff7eb-0899-41e0-9202-d1dfd5d3a375/graceful-termination/0.log"